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ABSTRACT 


The  Combat  Control  System  (CCS)  construct  developed  using  a  systems 
engineering  approach,  when  implemented,  will  provide  significantly  increased  levels  of 
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a  more  rested  and  effective  crew,  increasing  safety  of  ship,  while  potentially  saving  the 
Navy  $41.7  million  per  year. 
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Ownership  Cost  (RTOC)  ensuring  affordability  of  the  next  generation  Submarine  CCS. 
Since  the  submarine  platforms  play  a  significant  role  in  the  theater  level  engagement 
chain,  the  submarine  combat  system  effectiveness  cannot  adversely  impact  the  success  of 
the  overall  theater  level  engagement  chain. 

A  central  theme  of  our  research  is  to  show  the  effects  that  lowering  combat 
system  manning  has  on  the  overall  effectiveness  of  the  submarine  engagement  chain.  To 
assess  the  submarine  combat  system  effectiveness,  this  project  evaluates  the  functional 
data  flow  through  the  detect  to  engage  scenarios  to  evaluate  the  changes  in  the  level  of 
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EXECUTIVE  SUMMARY 

The  Combat  Control  System  (CCS)  construct  developed  using  a  systems 
engineering  approach,  when  implemented,  will  provide  significantly  increased  levels  of 
automation  to  perform  strategic  and  tactical  missions  with  greater  platform  Situational 
Awareness  (SA),  contributing  to  the  success  of  the  overall  theater  level  kill  chain.  This 
greater  level  of  automation  allows  for  a  reduction  in  manpower  from  48  sailors  in  the 
current  Virginia  Class  operational  base-line  to  23  sailors,  with  four  CCS  operators  per 
shift  and  an  average  utilization  of  slightly  greater  than  34.1%.  This  52%  reduction  in 
manpower  utilization  will  provide  a  more  rested  and  effective  crew,  while  increasing 
safety  of  ship  and  potentially  saving  the  Navy  $41.7  million  per  year. 

One  current  thrust  for  the  technical  community  within  the  United  States  Navy 
Submarine  Force  is  how  the  technical  community  can  sensibly  implement  Reduced  Total 
Ownership  Cost  (RTOC)  ensuring  affordability  of  the  next  generation  Submarine  CCS. 
One  of  the  important  operational  cost  drivers  is  manpower,  which  can  be  reduced  by 
decreasing  the  number  of  personnel  required  to  operate  the  Submarine  CCS.  However, 
reductions  in  CCS  personnel  must  be  accomplished  while  maintaining  the  combat  system 
effectiveness.  Since  the  submarine  platforms  play  a  significant  role  in  the  overall  success 
of  the  theater  missions  the  submarine  combat  system  effectiveness  cannot  adversely 
impact  the  overall  theater  level  engagement  chain.  Due  to  the  nature  of  the  submarine 
missions,  the  Rules  of  Engagement  (ROE)  may  not  require  a  target  “kill”  but  may  require 
engagement  in  a  role  to  track  for  the  purposes  of  monitoring.  For  this  reason  the 
submarine  engagement  chain  can  perform  the  tracking  or  kill  depending  on  the  ROE. 

There  are  many  gauges  by  which  to  measure  Total  Ownership  Cost  (TOC)  but  the 
single  largest  driver  is  the  Direct  Unit  Cost  (DUC)  associated  with  manning  [GAO,  2003] 
[Allison,  2000],  The  current  NAVSEA  focus  in  advancing  existing  individual  subsystems 
and  low-level  tools  has  not  actively  considered  decreasing  manpower,  but  rather 
increasing  and  improving  system  capabilities  for  the  operator  to  conduct  various  aspects 
of  tactical  missions. 

To  address  the  feasibility  of  reduced  manning  a  new  systems  engineering 


IX 


approach  and  Concept  of  Employment  (CONEMP)  was  developed  to  support  the  current 
platform  level  engagement  chain.  To  determine  the  feasibility  of  this  new  construct  in 
reducing  CCS  personnel,  a  system  model  was  developed  and  evaluated  by  varying  levels 
of  man  versus  machine,  the  number  of  men  supporting  the  CCS  and  the  effectiveness  of 
the  combination  of  software  algorithms  and  machine  power. 
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I 


INTRODUCTION 


The  military  importance  of  the  United  States  Navy  (USN)  submarine  force  has 
been  articulated  in  the  Congressional  Research  Service  (CRS),  Congressional  Budget 
Office  (CBO)  and  Government  Accountability  Office  (GAO)  reports  [Navy,  2005] 
[CBO,  2002]  [GAO,  2001].  The  military  value  of  submarines  has  long  been  recognized 
by  foreign  military  leaders.  This  strengthens  the  need  to  retain  a  sustainable  submarine 
force  and  subsequently  a  capable  Combat  Control  System  (CCS)  [China  Naval 
Modernization,  2009], 

With  the  cost  of  construction  for  the  fast  attack  and  ballistic  missile  submarines 
receiving  the  attention  of  Congress,  the  overall  Life  Cycle  Cost  model  needs  to  be 
examined  to  ensure  appropriate  construction,  affordable  combat  system  acquisition  and 
reasonable  operational  suitability  costs  [Navy,  2010].  Highly  capable  and  affordable 
combat  systems  are  key  components  to  ensure  realistic  submarine  construction  and 
operations  costs. 

Due  to  the  sensitive  nature  of  this  project,  all  values  used  in  the  analysis  were 
derived  from  open  source  documentation.  These  assumptions  are  used  throughout  the  text 
and  are  listed  in  Appendix  B.  As  this  report  is  an  academic  exercise,  it  is  intended  to 
demonstrate  the  team’s  ability  to  plan  and  execute  a  project  from  a  systems  engineering 
perspective.  This  report  is  intended  to  provide  a  construct  from  which  a  separate  and  fully 
implementable  project  can  be  developed  utilizing  classified  parameters  and  values. 

A.  SUBMARINE  COMBAT  SYSTEM  ENGINEERING  PROJECT 

The  Submarine  Combat  System  Engineering  Project  (SCSEP)  is  an  information 
processing  system  that  translates  raw  sensor  data  into  actionable  information  used  to 
convey  an  understanding  of  the  operational  environment  to  the  supervisors.  The  combat 
system  is  comprised  of  both  the  machines  that  process  the  data  and  the  operators  that 
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control  the  machine  and  translate  the  data  into  information.  The  present  Concept  of 
Operations  (CONOPS)  for  the  legacy  combat  system  requires  a  contingent  of  specialized, 
well  trained  operators  to  gather  and  translate  the  sensor  data.  The  intent  is  not  to  replace 
the  CCS  operators  and  their  human  decision  making  process,  but  rather  to  realize 
efficiencies  in  machine -based  perception  and  comprehension  by  utilizing  the  processing 
power  of  computers  in  conjunction  with  people. 

B.  BACKGROUND  ON  THE  ENGAGEMENT  CHAIN 

A  central  theme  of  this  project  is  the  submarine  engagement  chain  illustrated  in 
Figure  1.  An  overview  of  the  engagement  chain  and  key  terminology  used  throughout 
this  report  is  included  to  provide  definitions  and  context  for  the  reader.  The  engagement 
chain  is  the  process  for  search,  detect,  identify,  track,  decide,  engage  and  assess  contacts 
in  the  environment.  As  depicted  in  Figure  1,  the  engagement  chain  is  based  on  a  modified 
version  of  Bloye’s  kill  chain,  which  is  defined  as  a  “Detect-Track-Identify-Approve- 
Launch-Control- Assess”  process  [Bio ye,  2009].  This  report  does  not  concern  itself  with 
the  actual  kill.  However,  it  focuses  on  the  on-board  processing  from  detection  through 
the  decisions  involved  with  engagement  rather  than  focusing  on  the  engagement  of 
contacts. 


Figure  1  Engagement  Chain 

The  term  sensor  is  used  throughout  this  report  and  defined  here  for  context. 
Sensors  are  devices  which  measure  energy  in  the  submarine  operating  environment  and 
converts  it  into  information  that  the  operator  can  evaluate  as  part  of  the  decision  making 
process.  Sensors  include  sonar,  imaging,  Electronic  Surveillance  (ES)  and 
communications.  These  sensors  passively  detect  contacts,  which  emit  acoustic, 
electromagnetic,  visual,  infrared,  and  radio  frequency  energy.  Assuming  that  active  sonar 
is  rarely  used  in  operations,  only  energy  received  by  passive  sensors  is  included.  A 
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contact  can  be  sensed  by  the  system  as  either  surface  or  subsurface. 


The  first  step  in  the  engagement  chain  is  the  search  process,  which  involves  the 
use  of  submarine  sensors  to  receive  energy  to  detect  contacts.  As  a  result  of  the  search 
process,  the  second  step,  detect  occurs  when  the  operator  validates  an  incoming  contact. 
The  challenge  for  the  combat  system  operator  is  to  recognize  and  discriminate  between 
contact  details,  such  as  Signal  to  Noise  Ratio  (SNR)  in  decibels  (dB),  bearing  and 
frequency  content.  The  sensing  of  energy  is  necessary,  but  not  sufficient,  to  assess  if 
there  is  a  successful  detection. 

The  third  step  in  the  engagement  chain  has  the  objective  of  identifying  the 
contacts  based  on  the  contacts’  acoustic  frequency  components  and  location  in  the 
environment.  Each  contact  is  also  uniquely  categorized  into  classes  (e.g.,  warship, 
merchant,  pleasure  craft  or  biologies)  by  its  frequency  content,  bearing,  bearing  rate  and 
range  rate.  Each  contact  is  assigned  a  unique  contact  number  to  differentiate  contacts 
within  the  combat  system. 

During  the  identification  process,  the  watch  team  monitors  the  categorized 
contacts  for  Contacts  of  Interest  (COI)  or  Contacts  of  Concern  (COC)  [DEVRON-12, 
2011].  A  COI  is  any  contact  that  requires  the  attention  of  the  contact  management  team. 
A  COC  is  any  contact  that  presents  a  risk  of  counter  detection,  hostile  engagement,  or 
collision  [DEVRON-12,  2011], 

A  COI  is  determined  by  specific  operational  tasking  or  the  potential  to  impact 
ownship  safety  or  security.  Contacts  that  are  identified  as  COIs  are  actively  monitored  by 
the  watch  team.  A  COI  is  elevated  to  a  COC  if  any  of  the  established  thresholds  are 
exceeded  [DEVRON-12,  2011], 

The  fourth  step  involves  initiating  the  track  process.  During  the  track  process,  the 
contact  management  team  uses  the  data  from  the  submarine  sensors  and  subsystems  to 
determine  and  maintain  the  course,  speed,  and  range  of  a  contact,  or  multiple  contacts. 
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The  information  gained  from  the  tracking  process  is  utilized  to  predict  collisions  or  to 
employ  the  weapons  system  to  develop  targeting  and  firing  parameters. 

The  fifth  step  is  the  decide  process  of  the  engagement  chain,  which  is  the 
responsibility  of  the  Commanding  Officer  (CO)  and  executed  by  the  Officer  of  the  Deck 
(OOD).  He  must  maintain  situational  awareness  (SA)  that  will  allow  him  to  decide  what 
action  is  required.  During  the  decide  stage,  the  OOD  must  make  a  decisive  action  to 
either  avoid  or  engage  a  COC.  If  the  OOD  decides  to  avoid,  the  submarine  is  actively 
maneuvered  by  changing  course,  speed,  or  depth.  The  sixth  step  is  the  engage  process 
when  the  CO  directs  the  employment  of  the  weapons  systems.  The  seventh  step  is  the 
assess  process  which  occurs  following  the  engagement.  To  assess,  the  submarine  sensors 
are  utilized  to  determine  the  outcome  of  the  engagement. 

C.  PROBLEM  STATEMENT 

There  are  many  gauges  by  which  to  measure  Total  Ownership  Cost  (TOC),  but 
the  single  largest  driver  is  manning  [GAO,  2003]  [Allison,  2000].  By  implementing 
Reduced  Total  Ownership  Cost  (RTOC)  within  the  USN  Submarine  Force  the  technical 
community  must  manage  the  affordability  of  the  next  generation  Submarine  CCS.  Since 
one  important  operational  cost  driver  is  manpower,  TOC  can  be  reduced  by  decreasing 
the  number  of  personnel  required  to  operate  the  Submarine  CCS. 

The  central  focus  of  our  research  was  on  how  to  lower  manning  in  light  of  the 
engagement  chain.  To  assess  submarine  combat  system  effectiveness,  this  project 
evaluated  the  data  flow  using  scenarios  to  evaluate  changes  in  system  parameters.  The 
project  also  looked  at  the  “man  versus  machine”  to  determine  the  feasibility  of  replacing 
personnel  with  automated  data  processing.  Reductions  in  CCS  personnel  must  be 
accomplished  while  maintaining  the  combat  system  effectiveness. 

The  current  submarine  combat  system  development  process  produces  products  in 
a  two-year  development  cycle  [Stevens,  2008],  This  process  enables  evolutionary 
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changes  to  each  subsystem,  and  has  realized  incremental  improvements  in  the  following 


areas: 


1.  Open  Architecture  (Networking):  Subsystems  have  been  networked 
together  to  provide  faster  sharing  of  information  with  hardware  independent  software 
[Stevens,  2008], 

2.  Sensors:  More  advanced  sensors  and  processing  capability  have  been 
added  [Stevens,  2008], 

3.  Automation:  Automated  trackers  and  “bell  ringers”  are  software 
algorithms  that  have  been  implemented  to  assist  the  operator  in  the  detection  and 
classification  of  acoustic  energy  or  visual  contacts  [Zamich,  2006],  Bell  ringers  are 
preset  thresholds  that  alert  the  operator  when  the  threshold  has  been  exceeded  The 
improved  system  performance  is  alluded  to  by  Stevens  when  he  points  out  that  the  sonar 
system  can  turn  a  sonarman  into  an  expert  if  they  are  looking  at  the  right  display  and  can 
interpret  a  quiet  diesel  submarine  from  noisy  merchant  vessels  [Stevens  2008]. 

While  Stevens  made  it  clear  that  these  networked  subsystems,  advanced  sensors 
and  improved  automated  tools  have  increased  the  operator's  success  detecting  contacts,  it 
has  not  necessarily  decreased  their  workload  [Stevens  2008].  These  improvements  in  the 
systems  gives  the  operator  more  capability  at  detecting  contacts,  but  not  necessarily  a 
focus  on  applying  automation  to  reduce  the  quantity  of  operators  [Stevens  2008]. 

The  downside  of  this  evolutionary  process  is  that  even  with  the  intent  to  eliminate 
stove-piped  programs,  each  subsystem  remains  developed  as  part  of  independent 
modernization  programs.  [Toth,  2010]  Therefore,  it  is  not  the  performance  of  the  sensors 
and/or  automation  of  the  existing  stove-piped  processes  that  needs  to  be  considered,  but 
rather  the  method  by  which  the  resultant  data  is  utilized. 

An  examination  of  the  platform-level  engagement  chain  steps  (functions)  is 
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needed  to  reevaluate  the  workflow  and  develop  a  new  CCS  CONEMP  to  reduce 
manning.  Reduced  manning  levels  must  be  reached  without  degradation  in  the  overall 
submarine  effectiveness.  This  new  approach  to  workflow  will  require  a  new  construct 
that  defines  how  subsystems  come  together  to  form  a  viable,  tactical  CCS  system.  An 
assumption  of  this  project  is  that  a  paradigm  shift  will  be  required  in  the  way  the  systems 
are  utilized  and  organized,  resulting  in  higher  “people-ware”  efficiencies. 

D.  APPROACH  OVERVIEW 

The  purpose  of  this  project  was  to  develop  the  basic  concepts  of  an  architecture 
that  could  be  used  as  a  construct  for  the  next  generation  of  submarine  CCS.  This  report 
presents  a  method  of  reducing  the  manning  of  CCS  in  relationship  to  the  platform 
engagement  chain.  This  was  accomplished  by  developing  a  new  systematic  approach  to 
the  analysis  of  the  CCS  supporting  architecture  and  coupling  it  with  a  new  proposed 
CONEMP.  Scenarios  were  developed  that  represent  the  full  range  of  submarine 
operations.  From  these  scenarios,  a  stressing  case  was  created  to  demonstrate  that  an  end- 
to-end  model  based  on  the  engagement  chain  could  be  developed  for  the  submarine 
combat  system.  This  model  focuses  on  maintaining  mission  effectiveness  of  the 
submarine  CCS,  while  reducing  the  number  of  combat  system  personnel. 

To  illustrate  the  concept  of  the  project,  the  Department  of  Defense  Architectural 
Framework  (DoDAF)  Operational  View-1  (OV-1),  shown  in  Figure  2,  presents  the 
proposed  scenarios  for  the  project.  To  construct  a  system  model,  several  assumptions  and 
constraints  were  formulated  to  present  an  unclassified  treatment  of  the  project  and  create 
a  report  that  is  available  for  public  release.  These  scenarios  utilize  high  contact  density 
and  SA  to  evaluate  the  effectiveness  of  the  proposed  system. 
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Figure  2  SCSEP  OV-1 


E.  SCOPE 

The  scope  of  this  project  was  to  investigate  a  new  submarine  CCS  constmct  that 
could  reduce  TOC  and  maintain  or  improve  system  effectiveness.  A  high  level  system 
model  was  developed  to  support  mission  scenario  development,  capability  definition,  and 
engagement  chain  analysis.  The  results  include  recommendations  for  future 
development.  Sensor  inputs  and  performance  are  not  taken  into  account.  Sensors 
improvements  are  assumed  to  be  incrementally  advanced  under  their  own  program  paths. 
This  study  addresses  only  the  CCS  suite,  with  respect  to  RTOC  and  effectiveness,  as  it 
directly  relates  to  the  engagement  chain. 
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II 


SYSTEMS  ENGINEERING  PROCESS 


The  systems  engineering  process  section  details  the  structure  that  was 
implemented  in  the  SCSEP  to  define  the  problem,  identify  system  tradespace,  create  the 
system  model,  implement  simulations  and  conduct  an  evaluation  of  the  system  model. 

A.  SYSTEMS  ENGINEERING  PROCESS 

To  execute  the  SCSEP  project,  the  systems  engineering  (SE)  approach  detailed  in 
Figure  3  was  utilized.  This  approach  is  a  modification  of  the  International  Council  of 
Systems  Engineers  (INCOSE)  State,  Investigate,  Model,  Integrate,  Launch,  Assess  and 
Re-evaluate  (SIMILAR)  process,  and  has  been  adapted  to  the  scope  and  scale  of  the 
SCSEP. 


Figure  3  SCSEP  Systems  Engineering  Model 
a.  Refine  Problem 

Problem  refinement  involved  translating  stakeholder’s  needs  and  wants  into  a 
clear  problem  statement  with  prioritized  requirements.  The  significant  steps  of  this  phase 
were: 


Interview  Stakeholders  -  Stakeholders  were  interviewed  from  the  Naval  War 
College  (NWC),  Submarine  Development  Squadron  12  (DEVRON-12),  Navy  Submarine 
Medical  Research  Lab  (NMSRL)  and  the  Naval  Undersea  Warfare  Center  (NUWC)  to 
gather  subject  matter  information,  customer  wants  and  customer  needs. 

Develop  Mission  Scenarios  -  Scenarios  that  are  representative  of  plausible 
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submarine  missions  were  developed  based  on  stakeholder  needs.  These  scenarios  were 
used  to  outline  the  scope  of  the  requirements  and  to  evaluate  the  system  performance. 
The  mission  scenarios  provided  focus  and  the  initial  details  of  how  to  translate  the 
functions  and  submarine  engagement  chain  activities  into  the  overall  system 
requirements.  The  mission  scenarios  were  also  used  to  identify  the  maximum  expected 
contact  environment  for  the  simulations,  discussed  below,  as  a  stress  test.  This  was  done 
to  ensure  the  CCS  would  be  able  to  run  under  worse  case  loading. 

Generate  and  Analyze  Requirements  -  The  interviews  produced  a  shipboard 
functional  activities  task  list.  This  was  compared  to  the  stakeholder  wants  and  needs  to 
formulate  a  draft  requirements  list.  The  Universal  Navy  Task  List  (UNTL)  was  used  to  as 
a  source  for  the  tasks  identified  for  basic  submarine  operations,  anti-submarine  warfare, 
intelligence  gathering,  contact  tracking  and  weapons  employment. 

Refine  Requirements  -  The  initial  requirements  were  decomposed  into  the 
component  elements  of  the  engagement  chain.  Based  on  the  mission  scenarios  and  the 
analysis  of  model  outputs,  the  requirements  were  refined  to  achieve  the  best  combination 
of  system  parameters. 

Develop  Measures  of  Merit  (MOMs)  -  The  MOMs  are  the  full  set  of  Measures  of 
Effectiveness  (MOEs)  and  Measures  of  Performance  (MOPs)  derived  from  the 
stakeholder  inputs.  From  the  MOEs,  the  subset  of  MOPs  were  identified  to  categorize  the 
specific  performance  metrics.  Key  Performance  Parameters  (KPPs)  were  selected  out  of 
the  set  of  MOPs  as  being  the  most  important.  These  MOMs  are  used  to  measure  the 
goodness  of  the  design. 

Determine  the  relative  importance  weights  of  the  MOMs  -  With  MOMs  defined; 
the  impact  of  each  KPP  was  determined  and  assigned  a  relative  weighting  factor.  These 
weights  were  used  during  tradespace  evaluation  to  provide  some  quantitative  information 
about  the  stakeholders  needs  in  relation  to  the  possible  performance  issues  as  the 
tradespace  was  explored. 
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b.  Identify  Tradespace 


The  variables  for  the  overall  CCS  system  model  needed  to  be  identified  as  the 
basic  system  tradespace.  These  model  variables  required  adjustment  to  achieve  the  goal 
of  reducing  manpower  based  on  the  identified  requirements.  The  tradespace  consisted  of 
the  percent  man  versus  machine  used  at  each  workstation  and  the  total  number  of  men 
available.  The  overall  effectiveness  of  the  system  was  driven  by  the  combined 
effectiveness  of  man  and  machine  coupled  with  the  allocation  of  workload  (percent  man 
or  machine)  at  each  workstation  supporting  CCS. 

c.  Perform  Modeling 

The  modeling  phase  involved  developing  three  models  of  the  CCS: 

Capability  Based  Architectural  Model  -  Based  on  the  capabilities  and  functions 
identified  during  the  requirements  refining  process,  a  capabilities  based  architectural 
model  was  developed.  The  result  of  the  capability  based  architecture  was  used  as  the 
basis  of  the  simulation  model.  This  architectural  development  included  determining  and 
understanding  the  probabilities  of  the  contact  status  as  output  by  each  stage  along  the 
processing  path.  This  model  includes  the  tradespace  discussed  previously. 

Probability  Based  Simulation  Model  using  ExtendSim®  -  Extends im®  is  a 
modeling  and  simulation  application  that  was  used  to  simulate  the  overall  performance  of 
the  system  based  on  the  model  inputs.  This  model  represents  the  construct  for  the 
proposed  CCS  architecture.  Based  on  the  data  flows,  contact  attributes,  and  probabilities 
developed  in  the  capability  based  architectural  model,  a  separate  model  was  generated  to 
simulate  the  CCS.  The  model  was  used  to  provide  data  for  analysis  of  the  tradespace.  For 
the  simulations,  CCS  was  implemented  as  a  probabilistic  abstraction  of  the  CCS 
capabilities,  and  as  such,  represents  the  construct  for  the  proposed  CCS  architecture.  The 
output  of  the  probabilistic  model  was  used  to  compare  the  KPPs,  and  ultimately  any 
necessary  adjustments  that  needed  to  be  made  to  the  overall  system  requirements. 
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Cost  Model  -  A  cost  model  was  used  to  quantify  representative  savings  for  the 
report.  The  results  of  the  system  model  analysis  in  terms  of  personnel  reduction  will  be 
utilized  as  the  basis  to  determine  the  potential  savings  per  person,  per  platform.  The 
resulting  data  will  be  used  to  calculate  an  overall  yearly  and  potential  ten  year  savings. 
The  overall  intent  was  to  reflect  the  impact  of  system  manpower  reductions  as  a  function 
of  TOC. 


d.  Evaluate  Tradespace 

In  this  phase,  the  worst  case  scenario  was  used  to  stress  the  model.  The  results  of 
the  model  run  were  compared  to  the  system  requirements  and  the  variables  in  the 
tradespace  were  adjusted  as  necessary.  The  output  of  this  phase  is  a  recommended  set  of 
machine  constraints,  manpower  needs  and  “man  versus  machine”  utilization.  The  major 
steps  were  as  follows. 

Executed  Simulations  -ExtendSim®  software  was  used  to  develop  an  executable 
probabilistic  model  of  the  CCS  system.  This  executable  model  was  run  to  simulate  the 
performance  of  the  CCS  and  to  estimate  system  performance,  under  varying  operational 
parameters.  The  ExtendSim®  simulation  model  was  run  with  64  different  combinations 
of  input  parameters,  as  discussed  in  Appendix  C. 

Determined  Trade  Offs  -  The  team  performed  trade  off  analysis  by  analyzing  the 
output  of  the  simulation  runs.  The  output  of  the  probabilistic  model  was  used  to  compare 
the  KPPs  and  make  any  necessary  adjustments  to  the  model  variables  to  meet  the  overall 
system  requirements. 

e.  Final  Recommendation 

The  final  recommendation  identifies  the  way  ahead  for  the  CCS  system  based  on 
the  enabling  technologies,  potential  follow  on  work  and  organizational  changes  to 
support  fielding  the  CCS  system.  These  recommendations  were  based  on  the  results  of 
data  analysis  and  the  overall  conclusions. 
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Ill  STAKEHOLDER  ANALYSIS 


The  project  stakeholder  analysis  consists  of  two  parts  the  project  stakeholders  and 
the  submarine  CCS  stakeholders.  The  project  stakeholders  provided  contributions  to  the 
report  content  for  the  SCSEP  project.  The  submarine  CCS  stakeholders  are  the  subject 
matter  experts  that  sponsor,  acquire  or  use  the  SCSEP. 

A.  PROJECT  STAKEHOLDERS 

Project  stakeholders  include  the  student  team  members  from  Naval  Undersea 
Warfare  Center  Division  Newport  RI  (NUWCDIVNPT)  and  Naval  Surface  Warfare 
Center  Division  Carderock  Detachment  (NSWCCD)  Acoustic  Research  Detachment. 
Advisors  and  mentors  from  the  Naval  Postgraduate  School  (NPS)  and  NUWCDIVNPT 
also  provided  advice  and  feedback  during  the  course  of  the  project.  The  student  team 
members  performed  research,  conducted  modeling  and  analysis,  and  developed  this 
report.  During  the  project,  the  student  stakeholders  interacted  with  the  NPS  advisors 
weekly  to  provide  status  and  solicit  feedback.  Additionally,  the  NUWCDIVNPT  Chief 
Engineer  (CHENG)  served  as  a  stakeholder  and  provided  feedback  and  guidance  to  the 
team  periodically  during  the  conduct  of  the  project.  IPRs  were  conducted  that  gave  the 
opportunity  for  all  stakeholders  to  provide  feedback  and  recommendations  on  the  project. 
These  interactions  with  the  project  stakeholders  served  to  provide  real  time  feedback  to 
the  team  allowing  for  valuable  input  and  guidance  for  the  project.  Table  1  outlines  the 
major  project  stakeholders: 


12 


Name 

Community  of 

Practice 

Role 

Dr.  Jeffrey  Beach 

NPS 

Advisor 

John  Becker 

NSWCCD 

Project  Team  Member 

Shaun  Cookinham 

NUWCDIVNPT 

Project  Team  Member 

Shawn  Goode 

NUWCDIVNPT 

Project  Team  Member 

John  (Mike)  Green 

NPS 

Advisor 

David  Rhodes 

NUWCDIVNPT 

Project  Team  Member 

Denman  Sweetman 

NUWCDIVNPT 

Project  Team  Member 

David  Toth 

NUWCDIVNPT 

NUWCDIVNPT  CHENG 

Mark  Wasilewski 

NUWCDIVNPT 

Project  Team  Member 

Samuel  D.  Winograd 

NUWCDIVNPT 

Project  Team  Member 

Table  1.  Project  Stakeholders 

B.  STAKEHOLDERS  FOR  THE  SUBMARINE  COMBAT  CONTROL  SYSTEM 


Stakeholders  for  the  submarine  CCS  are  the  people  and  organizations  that  would 
be  the  real  world  acquirers,  developers,  and  users  of  the  submarine  CCS.  At  a  high  level 
these  stakeholders  fall  into  one  of  three  categories:  the  user  community,  resource 
sponsors,  and  the  acquisition  community. 


Name 

Subject  Matter  Expert 

(SME) 

Role 

DEVRON-12 

Submarine  CCS  Tactical 

Procedure  Development 

User  Community 

NWC 

Instruction  and  training  of  user 

NSMRL 

Psychological  and  Physiological 
analysis  of  users 

Former  Fleet  reps 

Submarine  CCS  user 

Resource  Sponsors 

Articles  on  Submarine  Cost 

Submarine  Warfare  Division 
(N87) 

Acquisition 

Community 

Student  Team  Members 

NAVSEA  PEO  SUBs 

Table  2  Stakeholders  for  the  Submarine  Combat  Control  System 
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1.  User  Community 

The  user  community  is  comprised  of  those  that  utilize  and  employ  the  system, 
instruct  and  train  the  users  to  employ  the  system,  as  well  as  study  and  monitor  the 
psychological  and  physiological  analysis  of  users. 

In  order  to  understand  the  needs  of  the  submarine  crew  that  utilize  and  employ  the 
system,  interviews  were  conducted  with  former  submariners,  as  well  as  representatives  of 
the  NWC,  NMSRL  and  DEVRON-12.  During  the  interviews  with  these  subject  matter 
experts,  there  were  five  common  themes:  data  confidence,  data  timeliness,  SA,  trust  of 
the  algorithms  and  the  limitations  of  human  operators. 

2.  Resource  Sponsors 

Resource  sponsor  requirements  were  defined  in  terms  of  MOMs  to  capture  the 
overall  needs  of  the  submarine  force.  The  establishment  of  the  MOMs  included  MOEs 
and  MOPs  which  reflect  the  importance  of  the  Fleet  needs.  The  RTOC  analysis 
performed  was  applicable  only  to  submarine  personnel  for  combat  control  and  attempt  to 
reduce  the  cost  of  the  CCS  to  the  Resource  Sponsor.  A  cost  model  was  used  to  illustrate 
the  cost  savings  that  could  be  realized  by  the  sponsor. 

3.  Acquisition  Community 

The  Program  Executive  Office  Submarines  (PEO  SUBs)  and  Program  Acquisition 
Resource  Managers  (PARMS)  at  the  Naval  Sea  Systems  Command  (NAVSEA)  comprise 
the  acquisition  community  for  the  submarine  platforms  and  the  CCS.  Each  team  member 
used  their  experience  from  years  of  supporting  the  PARM  in  technical  oversight  roles  and 
in  participation  in  planning  and  decision  meetings  to  provide  a  perspective  to  the  project 
from  the  acquisition  community. 
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IV  CONCEPT  OF  EMPLOYMENT 


The  proposed  CONEMP  is  designed  to  reduce  the  manpower  needs  for  the  CCS, 
while  maintaining  system  effectiveness. 

The  fundamental  system  model  of  the  CCS  is  shown  in  Figure  4.  This  model 
provides  context  for  the  construct  developed  in  this  report,  as  well  as  providing  context 
for  comparing  the  current  CCS  CONOPS  with  the  proposed  CONEMP.  In  this  systems 
model,  contact  energy  is  received  from  non-acoustic  sensors  (ES,  Automatic 
Identification  System  (AIS),  and  imaging)  as  well  as  acoustic  sensors  (sonar). 


SCSEP  Fundamental  System  Model 


Contacts 

•  SONAR 

*  Imaging 

*  ES 

•  AIS 


SCSEP 


Decision 

*  Engage 

*  Avoid 

*  Track 


CCS  Personnel 


Figure  4  SCSEP  Fundamental  System  Model 

Once  sensor  data  is  received,  the  operators  must  interpret,  and  correlate  the  data 
from  the  multitude  of  sensors  into  information  that  can  then  be  provided  to  the 
supervisors,  allowing  them  to  form  a  SA  picture.  Forming  and  maintaining  the  SA  picture 
with  the  current  CONEMP  is  operator  intensive. 
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The  intent  of  the  proposed  CONEMP  is  to  increase  utilization  of  a  machine-based 
infrastructure  (algorithms,  data  processing,  heuristics  and  machine  learning)  to  process 
the  received  acoustic  and  non-acoustic  data  as  though  it  were  a  compliment  of  expert 
specialized  operators.  To  achieve  an  expert  level  of  understanding,  the  lowest  level  data 
is  assembled  and  analyzed  using  Endsley’s  SA  model  [Endsley,  2006],  The  data  must  be 
recognized  and  understood,  and  a  solution  provided  to  the  decision  makers.  The  role 
change  proposed  for  the  SCSEP  operators  will  transform  them  into  data  verifiers,  as 
opposed  to  their  current  role  as  data  gatherers.  The  data  processing  algorithms  must 
effectively  present  the  correlated  data  as  information  to  the  operators.  The  operators  then 
verify  the  information  and  present  the  information  to  the  supervisors  to  decide  and  act 
upon.  The  intent  is  not  to  replace  the  human  decision  making  process,  but  rather  to 
realize  efficiencies  in  perception  and  comprehension  by  utilizing  the  processing  power  of 
computers  in  conjunction  with  people.  In  this  paradigm,  the  sensor  operators  are 
personnel  who  have  collective  Knowledge,  Skills,  and  Abilities  (KSA)  from  each  of  the 
CCS  system  elements  that  enable  them  to  assess  and  communicate  the  threat  profile  to 
the  supervisory  structure.  The  employment  of  the  particular  capabilities  is  dependent 
upon  the  intended  mission,  so  the  operator  must  be  knowledgeable  in  analyzing  the 
output  of  various  sensors. 

For  this  project,  three  mission-based  scenarios  were  developed,  and  then  merged 
into  a  single  worst  case  scenario.  During  these  major  shipboard  evolutions,  the  SCSEP 
model  provided  an  assessment  of  the  received  sensor  data  and  prioritized  the  threats  as  a 
function  of  range  and  target  motion.  If  the  model  indicates  a  probable  collision,  the  CCS 
operators  are  notified  and  appropriate  corrective  action  can  be  taken. 

These  three  scenarios  were  used  in  conjunction  with  the  proposed  CONEMP  to 
assess  the  models  developed  for  the  project  to  ensure  the  concept  was  modeled  correctly. 
The  level  of  man  and  machine  was  then  varied  to  compare  the  effectiveness  of  the  current 
CONEMP  with  varying  implementations  of  the  proposed  CONEMP. 
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A.  SCENARIO  CONSTRUCTION 


Representative  scenarios  were  generated  in  order  to  frame  and  bound  the 
conceptual  idea,  and  to  provide  a  context  for  development.  They  focused  on  Ship 
Safety/Self  Protect  functionality  and  maintenance  of  local  operational  SA  from  on-board 
sensors  and  off-board  intelligence. 

The  basic  mission  flow  consisted  of  receiving  tasking  from  the  theater 
commander.  The  platform  then  transited  to  the  identified  mission  area  while  maintaining 
SA  and  contact  avoidance.  While  in  transit  and  on  mission,  the  combat  system  team 
continuously  maintained  the  SA  necessary  to  convey  to  the  command  structure  all 
impending  issues  in  a  prioritized  manner.  The  SA  was  conducted  by  the  use  of  range, 
contact  type  and  course/speed  parameters  to  assess  the  potential  threat  picture.  The  stages 
of  the  operational  sequence  diagram  that  was  derived  to  develop  the  mission  scenarios 
are  shown  in  Figure  5  and  defined  in  this  section. 

From  the  transit,  the  platform  is  tasked  to  execute  the  mission.  Each  mission 
requires  the  detection,  identification,  tracking,  decision  making,  engagement  or 
avoidance  and,  if  applicable,  the  assessment  of  the  engagement.  For  each  of  the  three 
scenarios  analyzed  the  execution  components  for  the  scenario  are  identified. 
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Figure  5  Mission  Scenario  Construct 

The  following  sections  outline  an  overview  of  each  of  the  three  scenarios  used. 
Each  scenario  is  assumed  to  consist  of  port  egress  and  transit  on  the  surface  until  the  dive 
point.  The  surface  transit  presents  the  case  where  all  of  the  sensors  are  utilized,  including 
sonar,  imaging,  ES  and  AIS.  The  anticipated  surface  transit  contact  loading  has  the 
potential  to  receive  significant  numbers  of  simultaneous  contacts.  The  task  for  the  combat 
system  personnel  during  the  transit  is  to  deconflict  the  tactical  picture  and  ensure  that  the 
command  structure  has  an  accurate  representation  of  the  situation. 

Similarly,  during  each  mission  deployment  it  is  assumed  that  the  platform  must 
surface  to  Periscope  Depth  (PD)  either  to  support  the  mission  directly  or  to  receive  radio 
broadcast  messages  for  tasking  assignments  and  Rules  of  Engagement  (ROE).  These 
evolutions  present  cases  where  all  sensors  are  simultaneously  receiving  and  processing 
contact  data  to  reconcile  the  tactical  picture.  As  in  the  transit  case,  the  combat  system 
personnel  must  deconflict  the  fused  contact  data  to  ensure  an  accurate  representation  of 
the  tactical  picture  is  presented  to  the  command  personnel. 
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1.  Scenario  1:  Undersea  Warfare 

Search,  Detect,  Identify,  Track,  Decide  and  Engage  Hostile  Combatants 

This  scenario  consists  of  the  ownship  ability  to  locate,  track  and  potentially 
engage  hostile  combatants  (either  surface  or  submarine)  in  actual  combat.  While 
underway,  the  sonar  sensors  make  an  initial  analysis  of  the  environment  to  establish  the 
contact  density  and  attempt  to  identify  the  COL  The  system  provides  a  means  to  track  the 
various  contact  features.  The  sensor  operators  in  their  new  role  receive  the  sensor  (sonar 
or  all  depending  upon  depth)  data  and  assembles  a  comprehensive  situational  picture.  In 
the  event  that  a  COI  transitions  to  a  COC,  decisions  must  be  made  depending  on  the 
ROE.  Ultimately,  the  decision  must  be  made  to  track  or  engage. 

If  the  decision  is  made  to  engage,  a  fire  control  solution  is  calculated  for  the 
weapon  system(s).  The  torpedo  room  is  then  made  ready.  Upon  firing  of  weapons,  the 
command  and  combat  system  operators  confirm  the  level  of  success  against  the  target. 
For  surface  targets,  periscope  imagery  is  utilized.  Sonar  contact  assessment  is  used  to 
gauge  the  level  of  success  against  submerged  targets  by  verifying  the  absence  of  the  prior 
acoustic  features,  as  well  as  the  presence  of  acoustic  information  related  to  target  damage 
or  destruction. 

Conversely,  if  the  decision  is  made  to  track,  pursue,  or  avoid,  SA  must  be 
maintained  for  the  COC  until  it  is  deemed  no  longer  a  concern.  In  the  case  of  a 
submerged  COC,  the  sonar  sensors  are  utilized  to  analyze  the  acoustic  signature  and 
monitor  the  system  audio  and  to  recognize  the  acoustic  characteristics  that  are  consistent 
with  a  hostile  combatant.  The  ownship  watch  bill  must  ascertain  the  likelihood  of  the 
launch  of  threat  torpedoes  and  determine  evasion  tactics  and  countermeasures.  In  the  case 
of  a  surface  COC,  both  acoustic  and  visual  sensors  may  be  utilized.  At  all  times,  the  crew 
must  maintain  SA  for  both  the  COC,  as  well  as  any  COI. 

The  flow  for  this  scenario  follows  the  engagement  chain.  The  sensors  receive 
acoustic  energy,  imagery  or  ES  depending  upon  ownship  depth.  The  detection  algorithms 
process  the  data  in  the  form  of  initial  contact  reports.  The  contact  reports  are  analyzed  to 
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identify  the  frequency  contents  of  data.  This  interrogation  of  the  data  is  performed  to 
identify  the  contact  as  a  COI,  COC,  or  no  threat. 

The  classification  of  the  acoustic,  image  or  ES  signatures  for  each  contact  allow 
the  operator  to  categorize  a  contact  as  a  specific  vessel  utilizing  acoustic  or  visual  means. 
The  classification  is  in  the  form  of  warships  (surface  or  submarines),  pleasure  craft, 
merchant  vessels  or  biologies.  Once  a  contact  is  properly  categorized,  the  process  of 
tracking  it  through  bearing  space  is  initiated.  For  COI  and  COC,  an  automated  tracker  is 
initialized.  The  bearing  updates  are  processed  through  Target  Motion  Analysis  (TMA) 
on  all  contacts.  The  contact  management  team  is  alerted  when  characteristics  within  the 
system  ascertain  a  potential  collision  or  significant  threat  posture  exists.  The  tracking 
process  can  continue  for  great  lengths  of  time  depending  upon  the  tasked  mission.  The 
system  also  must  determine  the  threat  posture  based  on  the  ROE.  If  the  mission  presents 
the  potential  of  hostile  threats,  the  system  posture  must  be  set  to  analyze  clues  and  report 
out  threat  conditions  without  operator  interaction. 

(a)  Decisions  are  necessary  based  upon  the  recommendations.  In  the  case  of  an 
active  engagement,  the  Fire  Control  solutions  are  calculated  for  COCs  and  the 
ROE  are  verified.  Then  the  CO  must  determine  whether  to  engage,  avoid  or  track 
the  contact. 

(b)  To  engage,  the  Fire  Control  solutions  are  sent  to  Weapons  Control  (WC)  for 
programming  weapons.  The  COC  TMA  solution  (range  and  bearing)  estimates 
are  used  to  establish  the  final  firing  solution.  The  system  operator  preprograms 
the  weapons  and  seeks  permission  to  fire  weapon(s). 

2.  Scenario  2:  Special  Operations  Force  Delivery 
Covert  Entry  into  Hostile  Waters 

This  scenario  consists  of  a  need  to  deliver  SOF  covertly  into  hostile  waters.  To 
achieve  the  objective  of  the  mission  requires  comprehensive  SA  and  understanding  of  the 
tactical  picture.  There  is  reliance  on  sonar,  radar,  imagery  and  satellite  communications 
for  surveillance.  Ownship  will  enter  shallow  water,  release  a  Sea,  Air,  and  Land  (SEAL) 
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team  and  covertly  exit  the  area.  The  model  used  for  this  scenario  includes  the  need  to 
cross  a  shipping  lane  to  present  the  system  with  a  high  contact  density  problem,  which 
stresses  the  system  operators  contact  TMA,  and  data  fusion  aspect. 

The  primary  objective  is  to  deliver  the  SEAL  team  and  retrieve  them  upon 
completion  of  their  mission.  Following  the  SEAL  team  retrieval  the  ownship  platform 
must  return  through  the  high  contact  density  shipping  lane.. 

The  flow  of  this  scenario  is  as  follows:  Ownship  must  enter  the  Mission 
Operating  Area.  To  achieve  covert  entry,  the  system  must  accurately  search,  detect, 
identify  and  track  all  contacts  within  the  search  space.  The  operators  must  trust  the 
contact  information  and  understand  the  consequences  of  being  discovered.  While 
ownship  is  deep,  sensor  operators  receive  and  prioritize  all  initial  contact  reports  based  on 
the  characteristics.  While  the  submarine  is  crossing  a  shipping  lane  and  deep,  sensor 
operators  receive  and  prioritize  all  initial  contact  reports.  When  in  the  littorals  and  at  PD, 
sonar  is  used  in  conjunction  with  imagery  and  ES  to  verify  the  contact  profile. 

Once  a  vessel  is  properly  categorized,  the  process  of  tracking  it  through  the  use  of 
bearings  only  TMA.  For  COI  and  COC,  a  means  to  track  the  contact  is  assigned  via 
automated  tracking.  The  bearing  updates  are  processed  through  TMA  on  all  contacts. 
The  contact  management  team  is  alerted  when  characteristics  within  the  system  ascertain 
a  potential  collision  or  significant  threat  posture  exists.  The  tracking  process  can 
continue  for  great  lengths  of  time  depending  upon  the  tasked  mission.  The  system  also 
must  determine  the  threat  posture  based  on  the  ROE.  If  the  mission  presents  the  potential 
of  hostile  threats,  the  system  posture  must  be  set  to  analyze  clues  and  report  out  threat 
conditions  without  operator  interaction. 

To  execute  the  mission,  the  navigator  must  locate  the  intended  shore-based 
stations,  determine  best  landing  site,  and  request  that  ownship  enter  shallow  water.  The 
SEAL  team  is  released  to  perform  their  mission.  The  submarine  either  exits  the  operating 
area  or  hovers  until  SEAL  team  retrieval.  Once  the  SEAL  team  has  been  successfully 
recovered,  the  submarine  exits  the  mission  area. 
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3.  Scenario  3:  Advance  Battlespace  Preparation 

Carrier  Strike  Group  Support 

This  scenario  provides  a  forward  presence  to  the  ISR  to  support  air  operations. 
This  scenario  dictates  a  need  for  covert  identification  of  all  vessels  in  the  operating  area. 
Various  sensors  are  utilized  during  this  scenario;  sonar,  periscope,  satellite  imagery  and 
intelligence.  The  operators  are  required  to  search  the  operating  area  to  ensure  safe  carrier 
operations.  Full  tactical  control  and  understanding  of  the  environment  are  required  to 
assemble  SA  so  that  accurate  tactical  picture  data  will  be  transmitted  back  to  Fleet 
Commands. 

The  flow  for  this  scenario  follows  the  engagement  chain  and  is  as  follows:  While 
the  system  is  in  search  mode,  the  sensors  receive  energy  in  the  form  of  acoustic  (via 
sonar),  Imagery,  or  ES  depending  upon  ownship's  position  in  the  water  column.  In  the 
detect  stage,  algorithms  process  the  data  in  the  form  of  initial  contact  reports.  The 
contact  reports  are  analyzed  in  the  identification  stage  for  frequency  contents  of  the  data. 
This  interrogation  of  the  data  is  performed  to  classify  the  contact  as  a  COI,  COC,  or  no 
threat.  The  track  stage  utilizes  the  contact  reports  as  well  as  the  frequency  contents  to 
maintain  contact  location  and  kinematics.  The  decide  stage  uses  the  tracker  outputs  and 
other  data  (other  sensors  as  well  as  contact  reports,  etc)  to  assemble  the  SA  picture, 
enabling  correct  decisions  by  the  crew.  Engagement  decisions  require  an  understanding 
of  SA,  as  well  as  mission  requirements. 

The  classification  of  the  acoustic,  image  or  ES  signatures  for  each  contact  allow 
the  operator  to  identify  a  contact  into  a  specific  vessel  category  utilizing  acoustic, 
electronic  or  visual  means.  The  classification  is  in  the  form  of  Warships  (Surface  or 
Submarines),  pleasure  craft,  merchant  vessels  or  biologies.  Once  a  contact  is  properly 
categorized,  the  process  of  tracking  the  contacts  through  bearing  space  is  initiated.  For 
COI  and  COC  a  means  to  track  the  contact  is  assigned  via  automated  tracking.  The 
bearing  updates  are  processed  through  TMA  on  all  contacts.  The  contact  management 
team  is  alerted  when  characteristics  within  the  system  ascertain  a  potential  collision  or 
significant  threat  posture  exists.  The  tracking  process  can  continue  for  great  lengths  of 
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time  depending  upon  the  tasked  mission.  The  system  must  also  determine  the  threat 
posture  based  on  the  ROE.  If  the  mission  presents  the  potential  of  hostile  threats,  the 
system  posture  must  be  set  to  analyze  clues  and  report  out  threat  conditions  without 
operator  interaction. 

(a)  Decisions  are  necessary  based  upon  the  recommendations.  In  the  case  of  an 
active  engagement,  the  Fire  Control  solutions  are  calculated  for  COCs  and  the 
ROE  are  verified.  Based  on  the  mission  objective  of  ISR,  the  OOD  must 
determine  to  avoid  or  track/pursue,  and  as  a  last  resort  engage  the  contact. 

(b)  Periodically,  the  surfacing  to  PD  is  necessary  to  communicate  the  results  of 
the  mission  with  the  Fleet  commanders  and  to  receive  further  tasking  and 
intelligence  information. 

B.  MISSION  EFFECTIVENESS,  AND  NEED  FOR  SITUATIONAL 
AWARENESS 

The  ultimate  goal  of  the  submarine  is  to  complete  the  assigned  mission.  Clearly, 
this  has  to  do  with  completing  mission  specific  goals  assigned  by  the  Navy.  However,  in 
order  to  accomplish  these  goals  there  is  a  set  of  basic  requirements  that  have  to  be 
completed  as  prerequisites.  These  basic  requirements  can  be  summarized  as  maintenance 
of  safety  of  ship  and  minimizing  vulnerability  [Bundy,  2010].  This  is  depicted  in  Figure 
6. 


Mission  Effectiveness 
Reduce  Vulnerability 

Ship  Safety  /  Self-Protect 


Figure  6  Basic  Mission  Effectiveness  Requirements  Hierarchy 


As  mentioned  previously,  for  the  purposes  of  this  project,  many  of  the  ancillary 
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activities  that  occur  in  the  day-to-day  workings  of  the  submarine  are  ignored  in  favor  of 
only  those  tasks  that  take  place  in  the  CCS  related  to  the  engagement  chain. 

C.  SITUATIONAL  AWARENESS 

The  overall  progression  of  tasks  in  the  combat  system  is  directly  related  to  the 
submarine  warfare  kill  chain.  Figure  7  depicts  a  modified  engagement  chain,  which 
shows  that  the  overall  goal  is  typically  broader  than  a  simple  kill  and  encompasses  many 
different  mission  scenarios.  For  the  rest  of  the  report  it  will  be  assumed  that  search  was 
successful  and  the  engagement  chain  diagrams  will  omit  the  search  step 

A  necessary  requirement  for  maintaining  the  safety  of  the  submarine  is  situational 
awareness.  SA  is  basically  a  matter  of  comprehending  the  environment:  if  you  do  not 
understand  your  environment,  you  cannot  maintain  safety,  and  likely  not  stealth. 
Understanding,  both  the  static  (landmasses,  undersea  features)  and  the  dynamic  (other 
ships  in  the  area  that  might  affect  us)  environment  is  a  key  step  in  assembling  the  SA 
picture.  Further,  comprehension  of  the  environment  also  enables  good  decision  making 
about  what  actions  to  take  to  complete  the  mission  successfully,  including  everything 
from  how  to  steer  the  submarine  to  completing  mission  tasks.  For  purposes  of  this  report, 
we  will  use  the  following  definition  of  SA:  “The  perception  of  the  elements  in  the 
environment  within  a  volume  of  time  and  space,  the  comprehension  of  their  meaning,  and 
the  projection  of  their  status  in  the  near  future”  [Endsley,  2006]. 

One  key  product  from  the  discussions  with  the  SME’s  representing  the  user 
community  was  a  review  and  validation  of  the  engagement  chain  stages  shown  in  Figure 
7.  It  was  derived  from  interviews  with  retired  submarine  commanding  officers  [Bundy, 
2010]  [Pillsbury,  2010],  as  well  as  team  experience.  The  list  was  confirmed,  with  minor 
improvements,  by  representatives  from  DEVRON-12  [DEVRON-12,  2011],  This  ordered 
set  of  stages  forms  the  basis  of  the  engagement  chain  stages. 
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Figure  7  Engagement  Chain  Stages 

The  activities  that  make  up  SA  are  perceive,  comprehend  and  project.  These 
concepts  will  be  discussed  in  detail  in  following  sections  and  will  be  mapped  to  the 
engagement  chain. 

D.  CAPABILITY  AND  FUNCTIONAL  REQUIREMENTS  DECOMPOSITION 
AND  ALLOCATIONS 

The  task  list  in  Figure  7  formed  the  basis  of  the  decomposition  shown  in  Table  3. 
In  this  table,  the  following  pattern  is  used.  The  highest  (most  abstract)  level  is  the 
engagement  chain,  which  decomposes  into  the  Level  0  (LO)  tasks.  Not  all  these  tasks 
were  modeled,  as  they  were  deemed  too  detailed  and  difficult  to  model  and  did  not  add  to 
the  overall  study.  Only  the  LI  tasks  were  handled  in  the  model.  The  LO  tasks  could 
possibly  be  attempted  later. 
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Engagment  Chain  Stages  — ► 

Detect 

Identify 

Track 

Decide 

Engage 

Assess 

LO  Tasks  — ► 

Optimize  Sensors  for  Environment 

Filter  Data 

Look  for  Passive  Acoustic  Energy  Trends 

Look  for  Passive  Acoustic  Frequency  Trends 

Look  for  Passive  Electronic  Energy  Trends 

Look  for  Passive  Electronic  Frequency  Trends 

Perform  Visual  Sweeps 

Receive  Sensor  Data  (QUEUE) 

Analyze  Accoustic  tonal  structure  /  patterns 

Analyze  Electronic  tonal  structure  /  patterns 

Take  pictures 

Recognize  Contact  Highlights  from  pictures 

Perform  Data  Fusion 

Classify  Contact 

Identify  Contact 

Receive  AIS  signature 

Receive  Off-hull  Contact  Data 

Perform  Data  Fusion 

Receive  Ownship  Data 

Assign  /  maintain  Automated  Trackers 

Manually  track  through  acoustic  bearing 

Manually  track  through  electronic  bearing 

Perform  Visual  Tracking 

Develop  Contact  TMA  solution 

Predict  Collisions 

Predict  contact  loss 

Predict  Safe  Ownship  Maneuver 

Determine  best  Firing  Approach 

Determine  Weapon  Aiming  Data 

Perform  Data  Fusion 

Evaluate  Situation 

Set  New  Course 

Send  Targeting  data 

Preset  Weapons 

Maneuver 

Fire  Weapon 

Cue  Sensor 

Evaluate  success  of  engagement 

Implement  Directed  Search 

X 

X 

X 

X 

X 

X 

X 

Process  Acoustic  Data 

X 

X 

X 

X 

Process  ES  Data 

X 

X 

X 

X 

Process  Image  Data 

X 

X 

X 

Process  AES 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Assess  Acoustic  Data 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Assess  ES  Data 

X 

X 

X 

X 

X 

X 

X 

X 

Assess  Image  Data 

X 

X 

X 

X 

X 

X 

X 

X 

X 

H 

Queue  Contact  Data 

X 

X 

X 

X 

X 

X 

— 1 

Perform  TMA 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Fuse  Data 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Develop  SA 

X 

X 

X 

X 

Advise  to  Do  Appropriate  Thing 

X 

X 

X 

X 

Supervise  Doing  Appropriate  Thing 

X 

X 

X 

X 

Cue  Sensors  for  Directed  Search 

X 

X 

X 

X 

Evaluate  success  of  engagement 

X 

SONAR 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

ES 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

s 

o 

IMAGERY 
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X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

< 

Communications 
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X 

X 

X 

X 

X 

X 

X 

X 

X 

a 

TC 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

WC 

X 

X 

X 

X 

X 

Table  3  Functional  Decomposition  and  Initial  Allocation 
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The  second  part  of  Table  3  provides  the  functional  allocation  as  to  where  in  the 
system  each  of  the  function  takes  place.  Yellow  represents  the  sensors,  which  act  as 
service  providers  to  the  CCS,  providing  contact  data.  The  Tactical  Control  (TC)  and  WC 
listed  in  Table  3  represent  the  tactical  and  weapons  control  subsystems  of  the  CCS.  TC 
and  WC  are  not  implemented  in  the  simulations,  but  rather  are  noted  for  future 
consideration. 

E.  APPLYING  FITTS’ LIST 

One  of  the  initial  attempts  at  understanding  what  efficiencies  could  be  generated 
by  applying  automation  was  to  apply  Fitts’  List  to  each  of  the  tasks  in  Figure  7  [Fitts, 
1951],  While  Fitts’  List  is  somewhat  dated  compared  to  some  of  the  advances  in  newer 
computing  technologies  and  intelligent  machine  pattern  recognition,  it  was  deemed 
relevant  to  apply  the  rules  categorization  of  Fitts’  assertions  to  the  combat  control  sub¬ 
system.  The  practical  understanding  is  that  the  implementation  of  a  physical  system 
would  incorporate  more  modem  technologies  capable  of  discerning  contacts  in  a 
cluttered  environment.  According  to  Fitts’  list,  the  human  is  superior  at  perceiving 
patterns,  reasoning  inductively  and  exercising  judgment.  The  machine  can  help  in  the 
area  of  quick  response  and  deductive  reasoning.  As  long  as  judgment  is  left  to  the 
operator,  the  machine  as  a  decision  aid  can  be  helpful.  This  is  the  reasoning  behind  the 
statement  in  the  CONEMP  for  considering  adding  machine  learning  to  the  CCS 
algorithms. 


HUMANS  SURPASS  MACHINES  IN  THE: 


•  Ability  to  detect  small  amounts  of  visual  or  acoustic  energy 

•  Ability  to  perceive  patterns  of  light  or  sound 

•  Ability  to  improvise  and  use  flexible  procedures 

•  Ability  to  store  very  large  amounts  of  information  for  long  periods 

and  to  recall  relevant  facts  at  the  appropriate  time 

•  Ability  to  reason  inductively 

•  Ability  to  exercise  judgment 


MACHINES  SURPASS  HUMANS  IN  THE: 


*  Ability  to  respond  quickly  to  control  signal  s,  and  to  apply 

great  force  smoothly  and  precisely 

*  Ability  to  perform  repetitive,  routine  tasks 

*  Ability  to  store  information  briefly  and  then  to  erase  it  completely 

*  Ability  to  reason  deductively,  including  computational  ability 

*  Ability  to  handle  highly  complex  operations.  ie*  to  do  many  different 

things  ai  once. 
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Figure  8  Fitts'  List  [Schmidt,  2010] 


To  decide  if  the  thought  process  was  correct,  Fitts’  List  was  applied  to  the 
Combat  System  task  list.  Each  task  was  applied  to  a  corresponding  number  indicating  if 
people  do  better  (1),  machines  can  do  in  conjunction  with  human  guidance  (2),  a  machine 
can  perform  it  better  than  a  human  (3),  or  machines  should  be  able  to  do  it  better  than 
humans  (4).  Figure  7  was  modified  to  include  conformance  to  Fitts’  List,  as  presented  in 
Figure  9. 
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Figure  9  Fitts’  List  Applied  to  Combat  System  Tasks 

When  the  tasks  are  considered  in  this  manner,  it  seems  that  the  majority  of  the  tasks  on 
the  left  are  better  suited  towards  machines,  where  there  is  more  raw  data  to  process.  The 
tasks  more  towards  the  right  seem  better  suited  to  people.  There  are  fewer  tasks,  more 
processed  data  and  decisions  that  require  consideration  of  decision  consequences  (risk).. 

F.  KEEPING  THE  MAN  IN  THE  LOOP 

“The  Navy  is  a  service  of  custom  and  tradition,  and  ultimate  accountability” 
[Bundy,  2010],  What  this  meant  to  this  project  is  that  due  to  the  risk  of  loss  of  human 
life,  a  ground  rule  was  implemented  such  that  the  man  would  not  be  taken  out  of  the  loop 
completely.  Even  though  it  might  be  feasible  to  develop  an  entirely  unmanned  submarine, 
it  does  not  seem  likely,  given  risk  to  life  (collateral  damage  from  improperly  launched 
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weapons)  and  an  unprotected  payload  of  tactical  weapons.  The  socio-political 
international  consequences  of  being  wrong  on  a  contact  are  too  severe. 

The  fact  that  people  are  better  at  exercising  judgment  and  reasoning  deductively 
means  the  man  on  board  the  submarine  should  not  be  taken  out  of  the  decision  loop 
completely.  In  contrast,  an  automated  system  can  scan  many  data  sources  at  once, 
compute  all  possible  outputs  and  provide  suggestions  as  tactical  decision  aids.  For 
example,  automation  is  good  at  comparing  one  beam  to  the  next  to  identify  patterns 
between  them,  and  then  comparing  the  outputs  from  one  sensor  to  another  for 
consistencies  or  inconsistencies. 

Given  the  above  discussion,  the  team  decided  not  to  eliminate  the  human  aspect 
from  the  submarine,  but  rather  enhance  the  human  by  using  available  computing  power  to 
maximize  their  effectiveness.  One  of  the  first  concepts  to  come  out  of  this  line  of 
reasoning  is  that  the  project  team  needed  a  way  of  maximizing  the  utility  of  both  man  and 
machine,  playing  to  their  strengths,  which  might  ultimately  change  the  way  the  human 
does  his  job.  In  other  words,  the  team  realized  early  on  this  project  could  likely  cause 
changes  the  CCS  user’s  CONOPS.  This  mechanism  is  discussed  later  in  the  Man 
Machine  Tradespace  section. 
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G.  INTERFACES  AND  DATA  FLOW 


The  functionality  in  the  engagement  chain  provides  the  basis  of  the  interfaces  and 
the  data  flow.  As  previously  noted,  the  overall  engagement  chain  tasks  follow  a  SA 
progression.  In  general,  in  order  to  perform  the  tasks  to  complete  the  mission 
requirements,  the  first  step  is  to  perceive  the  environment.  The  three  phases  of  SA  can 
now  be  decomposed  into  individual  tasks  to  be  performed.  An  extremely  high  level  view 
of  the  model  is  presented  in  Figure  10.  This  figure  also  maps  the  SA  activities  to  the 
engagement  chain. 


a 

S.A. 

Perceive 

Comprehend 

Project 

Assess 

1  Sensor  > 
Outputs 


Figure  10  Simulation  Model  High-Level  View 

In  Figure  10  and  Figure  11,  the  colors  refer  to  how  these  functions  will  be 
implemented.  If  they  are  green,  either  man  or  machine  in  a  tradespace  can  perform  them. 
If  it  is  colored  red,  it  is  predominantly  human  performed,  although  there  may  be 
machines  acting  in  an  advisory  role.  Yellow  indicates  a  fully  automated  function.  The 
circles  represent  the  steps  of  the  subsequent  model  and  the  logical  mapping  of  the  model. 

The  first  step  in  the  process  shows  that  the  sensors  had  to  be  considered.  The 
sensors  chosen  for  this  report  are: 

•  Sonar,  which  is  broken  out  into  Broadband  and  Narrowband  data  streams, 

•  AIS,  which  is  a  transponder  system  not  unlike  Identified  Friend  or  Foe  (IFF), 

•  Electronic  Sensors,  which  detect  radar  systems, 
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Periscope  imagery. 


The  latter  three  can  only  be  used  when  the  submarine  is  at  PD,  so  that  was 
accounted  for  in  the  model  as  well.  Each  of  these  sensors  has  certain  range  capability  and 
different  output  certainties. 

The  longest-range  sensor  is  passive  sonar  (this  project  does  not  consider  active 
sonar).  The  problem  with  sonar  is  that  because  of  the  acoustic  nature  of  the  data  collected 
it  is  difficult  to  accurately  classify  and  practically  impossible  to  uniquely  identify  every 
contact.  Additionally,  under  normal  circumstances,  passive  sonar  does  not  provide  range 
accuracy,  and  cannot  differentiate  between  multiple  contacts  on  the  same  bearing.  In  this 
model,  sonar  is  broken  out  into  Broadband  and  Narrowband  views  of  the  data.  The 
Passive  Broadband  (PBB)  operator  uses  what  is  called  the  Broadband  Gram  (BBG).  The 
passive  narrowband  (PNB)  operator  uses  what  is  called  the  Narrowband  Gram  (NBG). 

Classification  can  be  done  partially  by  sonar,  but  it  is  often  aided  by  other  sensors. 
AIS  works  via  line  of  sight,  but  the  problem  is  that  warships  and  submarines  (highly 
critical  contacts)  typically  do  not  have  their  AIS  transponders  active. 

Periscope  imagery  can  be  used  to  determine  both  bearing  and  classification. 
Identification  and  range  of  the  contact  can  also  be  determined  dependent  on  the 
environment  and  other  variables. 

Electronics  (ES)  can  identify  the  type  of  radar  installed  on  a  contact.  With  some 
prior  knowledge,  this  can  also  help  in  the  identification  process. 

The  model  data  flow  can  now  be  decomposed  further  to  add  the  specific  sensor 
types,  as  shown  in  Figure  11. 
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Figure  11  Decomposed  Model  Data  Flow 

As  mentioned  previously,  for  a  successful  mission,  the  crew  needs  to  maintain 
safety  of  ship,  remain  stealthy,  and  complete  the  mission  requirements.  The  authors 
suggest  that  completing  the  mission  requirements  is  entirely  dependent  on  how  well  the 
SA  model  is  assembled  and  the  projection  of  the  current  situation  into  the  future. 
Successful  actions  depend  not  only  on  the  projection  of  SA,  but  also  on  the  mission 
requirements.  Since  the  mission  requirements  could  not  be  satisfactorily  modeled,  the 
actual  function  “Act”  is  not  modeled. 
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V  SYSTEM  REQUIREMENTS 


The  SCSEP  system  requirements  section  defines  the  MOMs  for  the  system.  These 
definitions  include  the  listing  of  the  MOMs,  MOEs,  MOPs  and  KPPs  along  with  the  description, 
metric  thresholds  and  goals,  appropriate  weighting  factors  and  rationale  on  how  the  metric  was 
developed. 

A.  MEASURES  OF  MERIT 

The  systems  requirements  for  this  project  are  reflected  in  the  MOMs,  shown  in  Table  4. 
These  MOMs  are  sufficient  to  execute  a  feasibility  study  to  reduce  manning  of  the  CCS,  while 
maintaining  system  effectiveness.  It  is  expected  that  any  future  work  to  implement  the  construct 
defined  herein  will  expand  this  list  of  MOMs,  as  required  for  full  system  acquisition.  The 
meaning  of  each  MOM  is  explained  below,  with  the  rationale  behind  each  provided  in  Table  4.  It 
is  important  to  note  that  the  thresholds,  goals  and  evaluation  weights  shown  in  Table  4  are 
assumed  based  on  the  project  members’  experience  within  their  respective  discipline  and 
engineering  judgments.  This  was  done  to  avoid  security  concerns,  and  to  keep  this  report  capable 
of  being  published  in  the  public  domain. 

1.  CC  Efficiency 

Efficiency  is  normally  defined  as  output  over  input.  CC  Efficiency  can  be  defined  to  be 
the  ratio  of  mission  effectiveness  to  the  cost  of  the  people  required  to  man  the  system.  The 
parameters  for  mission  effectiveness  are  defined  in  Table  4  CCS  MOMs.  This  form  of  cost  is 
just  one  measure  of  input,  and  discussed  in  a  later  section.  This  results  in  the  equation: 


(1) 


Where: 

CCE  -  CC  Efficiency  (relative) 

ME  -  Mission  Effectiveness 

Cost  -  Number  of  men  required  per  shift 
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MOEs 

MOPs 

©D 

Q 

P 

Q 

P 

KPP 

Threshold 

Goal 

Weight 

Project  Scorin 

Weight** 

* 

p 

o’ 

9 

1.0  cc 

Efficiency  * 

1.1  Mission 
Effectiveness* 


Percent 


Percent 


CC  Efficiency 

Mission 

Effectiveness/Nu 

mber  of  men  per 

shift 

50 

90 

N/A 

N/A 

86.9 

96.5 

0.7 

N/A 

Threshold  and  goals  based  on  weighted  roll-up  of  underlying  MOMs 

1.0  Maintain 

contact 

Percent 

maintained 

contacts 

Y 

80 

99.9 

0.25 

0.319 

1.1.1  through  1.1.3  were  balanced  to  ensure  that  safety  of  ship  would  be  the 

number  1  priority  in  mission  effectiveness,  maintaining  contacts  would  be  the 

second  priority,  as  it  is  common  to  both  safety  of  ship  and  mission  success,  and 

finally  mission  success  is  the  third  priority,  as  most  of  mission  success  is 

already  covered  in  1.1.1  and  1.1.2,  leaving  successful  completion  of  assigned 

tasking.  Though  this  might  seem  to  be  a  low  relative  rating  for  mission 

success,  Much  of  the  mission  success  of  a  submarine  is  beyond  the  control  of 

the  submarine  fleet.  Historical  statistical  data,  as  well  as  results  provided  by 

tactical  trainers  can  provide  this  type  of  data. 

Threshold  and  goals  of  mission  success  are  based  on  the  weighted  roll-up  of 

underlying  MOMs 

1.1.2  Maintain 

Safety  of  Ship  * 

•  Percent  of 

correct 

decisions 

•  Maintain 

Stand-off  zone 

Y 

95 

99 

0.6 

N/A 

1.1.3  Mission 

Success* 

Percent  of 

mission  success 

78 

93.4 

0.15 

N/A 

1.1. 3.1  Fusion 

Completeness 

of 

Comprehension 

•  Fusion  Score 

of  contributing 

sensors 

Y 

70 

95 

0.2 

0.038 

It  was  the  team  consensus  that,  although  important,  fusion  was  the  least 

important  contributor  to  ships  safety 

1. 1.3.2  TMA 

Correctness  of 

Comprehension 

•  Accuracy 

Y 

50 

90 

0.5 

0.096 

It  was  the  team  consensus  that  TMA  accuracy  was  the  major  contributor  to 

mission  success. 

1. 1.3.3 

Understanding 

90 

98 

0.3 

N/A 

Although  this  is  very  important  to  mission  success,  this  is  really  allocated  to 

man  as  supervisor  and,  as  such,  is  not  conducive  to  CCS  system  directly,  but 
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MOEs 


MOPs 


WD 

Q 

P 

Q 

P 

KPP 

Threshold 

Goal 

Weight 

Project  Scorin 

Weight** 

Rationale 

Mission 

Requirements  * 

more  to  the  full  combat  chain  of  command  at  a  very  high  level  of  a  system  of 

system  view. 

This  is  not  included  in  the  modeling,  as  how  to  quantify  the  “understanding 

mission  requirements”  was  beyond  the  knowledge  of  the  team  members,  and 

no  references  to  this  could  be  found. 

1.2  Cost 

Number 

of  men 

required 

per  shift 

Number  of  men 

required  per  shift 

Y 

8 

3 

.3 

0.547 

The  threshold  was  based  on  the  current  Virginia  (VA)  base-line,  while  the  goal 

of  three  was  based  on  the  team  feeling  uncomfortable  with  going  below  two 

sensor  operators  and  one  CCS  operator 

Table  4  CCS  MOMs 


MOMs  that  are  not  modeled  in  this  project. 

Adjusted  from  recommended  weights  to  compensate  for  those  MOMs  not  modeled. 
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2.  Mission  Effectiveness 

The  established  goal  was  to  maximize  system  effectiveness,  while  reducing 
manpower.  Mission  Effectiveness  is  a  function  of: 

•  Mission  success,  as  defined  by  the  mission  requirements, 

•  Ability  to  maintain  safety  of  ship, 

•  Maintain  contacts. 

Mission  success  is  partly  related  to  completeness  and  correctness  of 
understanding  the  tactical  picture  (environment)  in  addition  to  understanding  the  mission 
requirements.  Mission  success  is  measured  by  the  completeness  and  correctness  of 
comprehension  (data  fusion  and  TMA).  The  completeness  of  comprehension  is  the  fusion 
score  of  the  contributing  sensors  on  a  contact-by-contact  basis.  The  correctness  of 
comprehension  is  the  accuracy  of  TMA  on  a  contact-by-contact  basis.  Understanding 
mission  requirements  are  the  on-board  command  personnel’s  ability  to  comprehend  the 
mission  and  translate  the  requirements  into  a  mission  execution  plan.  For  the  purposes  of 
this  project,  the  understanding  of  the  mission  requirements  is  assumed  to  be  flawless. 

The  fusion  score  average  describes  the  fusion  comprehension  metric. 


(2) 


Where: 

CCF  -  Completeness  of  Comprehension  (Fusion) 

FSM  -  Fusion  Score  Mean 

The  correctness  of  the  modeled  comprehension  in  terms  of  TMA  is  composed  of 
the  fused  range  and  bearing  confidence  averages.  These  components  represent  the 
accurate  positional  locations  of  the  contacts  that  result  in  the  performance  of  the  tracker. 
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(3) 


Where: 

CCT  -  Correctness  of  Comprehension  (TMA) 

FBCM  -  Fused  Bearing  Confidence  Mean 
FRCM  -  Fused  Range  Confidence  Mean 
n  -  Number  of  contacts 

Ship  safety  consists  of  personal  practical  safety,  operational  risk  management  and 
general  safety  standards  [OPNAVINST,  2007],  Ship  safety  requires  certain  portions  of 
the  sensors  to  be  functional,  as  well  as  the  ability  to  fire  weapons.  Operational  risk 
management  is  considered  outside  the  project  scope  and  is  not  considered.  As  defined  in 
this  project,  maintenance  of  ship  safety  is  the  ability  of  the  crew  to  make  the  appropriate 
(Percent  of  Correct)  decisions  by  ensuring  that  navigation  accounts  for  actions  that  avoid 
collisions  and  provides  ample  warning  time  to  respond  with  evasive  maneuvers. 

Maintaining  contacts  is  the  ability  of  the  system  (which  includes  humans)  to  keep 
track  of  the  environment  and  vessels  in  it.  Vessels  managed  in  the  model  consisted  of 
warships,  submarines,  pleasure  craft  and  merchant  platforms.  Contact  Management  is 
measured  as  the  percent  of  maintained  contacts. 

The  percent  maintained  contacts  are  defined  as  the  ratio  of  the  sum  of  stale 
contacts  per  sensor  to  the  total  number  of  contacts  per  sensor.  Stale  contacts  are  contacts 
that  do  not  get  processed  by  a  sensor  within  a  preset  time  due  to  operator  workload. 
Further  definition  is  available  in  the  ExtendSim®  model  details  section. 


(4) 
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Where: 

CM  -  Contacts  Maintained  (%) 


3.  Manpower  Cost 

Cost  in  this  context  is  in  terms  of  personnel  per  shift,  and  represents  the  number 
of  personnel  involved  multiplied  by  the  utilization  coefficient.  These  factors  characterize 
the  use  of  the  personnel  in  the  TMA  and  the  total  pool,  or  the  distribution  of  the 
workload. 


(5) 


Where: 

NMen  tma  Pool  -  Number  TMA  Pool 
Utma  -  TMA  Utilization 
Npooi  -  Number  of  Men  Pool 
Upooi  -  Pool  Utilization 
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VI  MODEL  DESIGN 


During  the  modeling  phase,  a  model  of  the  CCS  was  developed  to  represent  the 
data  flow  between  the  major  CCS  functions  previously  derived.  This  data  flow  model 
implemented  a  queuing  theory  model  at  each  workstation  to  represent  how  well  the 
model  could  keep  up  with  incoming  contacts,  vice  departing  contacts. 

From  this  “paper”  model,  a  discrete-event  simulation  of  the  CCS  was 
implemented  using  ExtendSim®.  This  executable  simulation  served  as  the  basis  for 
exploring  the  tradespace.  It  supported  a  sensitivity  analysis  and  established  an  initial 
requirement  baseline  for  key  operational  parameters  (man  power  levels,  machine 
strength,  and  the  balance  between  man/machine  interactions  at  the  work  stations). 

In  order  to  explore  system  feasibility,  while  characterizing  system  operation,  it  is 
common  practice  to  model  a  process  as  a  series  of  probabilistic  decisions  [National 
Research  Council,  2007],  Thus,  for  this  project,  CCS  was  modeled  as  a  series  of 
functions,  whose  outputs  were  based  on  probabilistic  outcomes,  and  were  based  on  real 
world  physics  and  known  decision  processes. 

Figure  12  is  an  abstract  visual  aid  that  was  constructed  to  assist  in  describing  the 
approximate  level  of  data  within  the  engagement  chain.  This  aid  was  constructed  as  a 
rough  a  rough  correlation  to  the  number  of  people  required  to  sort  through  the  data.  The 
beige  area  shows  the  amount  of  raw  data  culled  through,  tapering  off  as  higher  level 
decisions  are  being  made.  Towards  the  right,  more  processed  data  outputs  are  required 
and  less  raw  data.  Currently,  there  are  more  people  at  the  lower  levels  working  on 
processing  that  data  (with  the  aid  of  computers).  Towards  the  right  side  of  the  chain, 
there  are  fewer  people,  but  they  are  making  higher-level  command  decisions.  Because  of 
the  risk  issues  mentioned  previously,  it  is  anticipated  that  the  decision-making  command 
structure  will  not  be  removed.  However,  there  is  an  underlying  supposition  that  driving 
towards  a  higher  level  of  automation  in  the  detecting,  identifying  and  tracking  tasks  will 
lower  the  manning  requirements  overall.  In  addition,  this  computational  chain  will  lead 
into  the  decisions,  where  automation  can  be  used  to  help  put  the  SA  picture  together  and 
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provide  suggestions  on  the  TMA  and  Data  Fusion  (combination  of  contact  information 
into  knowledge  of  that  contact).  This  automation  allows  the  machine  to  suggest  possible 
courses  of  action  to  the  command  personnel,  such  as  how  and  when  to  steer  the 
submarine,  but  leaving  the  decisions  to  the  human. 


Overloaded:  Need  more  people 
Lots  of  data  to  plow  through 


Need  situational 
awareness  and  command, 
not  people 


Amount  of  raw  data 


People  required 


Decision  making: 

Level  of  "I lig  ■  level  Hu  r>  -n  involvement 


Figure  12  Data  versus  Human  for  Decision 

The  final  architecture  of  the  combat  system  (which  includes  man  in  the  system) 
should  be  one  that  provides  the  best  processing  and  the  best  decision-making.  This 
balance  is  the  crux  of  the  model:  to  be  able  to  input  various  levels  of  man  and  machine 
for  certain  tasks,  and  find  empirically  through  simulation  the  most  optimum  levels. 

The  model  design,  and  hence,  the  high  level  architecture  is  based  on  the  definition 
of  SA:  Perceive,  comprehend  and  project  [Endsley,  2006]. 

1.  Perceive 

In  order  to  comprehend  the  environment,  the  first  step  is  to  sense  the  environment 
and  assess  those  sensor  outputs.  For  this  study,  the  number  of  operators  was  not  defined 
(although  Figure  13  shows  notional  workstations).  One  of  the  desired  outcomes  is  for  the 
model  to  help  determine  the  number  of  required  operators.  Although  it  is  outside  the 
scope  of  this  project  to  define  a  physical  architecture,  one  of  the  assumptions  made  was 
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that  each  operator  console  was  identical,  and  depending  on  the  job  performed,  the 
operator  simply  runs  a  different  piece  of  software.  This  is  like  a  home  computer.  If  you 
want  to  work  with  email,  you  activate  an  email  client.  If  you  wish  to  do  graphic  editing, 
you  activate  an  appropriate  application  for  that.  It  is  assumed  in  this  model  that  if  the 
operator  has  to  perform  Broadband  Gram  (BBG)  operations,  he  activates  the  BBG 
application  or  the  imagery  program  if  he  needs  to  scan  the  periscope. 

The  actual  algorithms  that  will  be  used  to  implement  the  perception  engine  are  not 
modeled  in  this  study.  However,  the  probabilities  of  the  various  outcomes  from  that 
processing  are  recorded.  The  assumption  is  made  that  the  operator  will  have  a  higher 
level  of  automation  than  is  currently  available  in  the  Fleet.  The  operator  is  still  primarily 
concerned  with  what  is  going  on  at  the  sensor  level;  that  is,  finding  contacts  and 
identifying  them.  The  model  is  therefore  designed  to  vary  the  level  of  automation, 
resulting  in  more  or  less  human  interaction,  and  more  or  less  system  effectiveness, 
yielding  more  or  less  cost. 

Depending  on  the  job  and  the  cognitive  loading  involved,  the  operator  may  be 
able  to  handle  multiple  tasks.  This  is  not  unlike  working  on  graphic  editing,  but 
occasionally  checking  email;  both  tasks  are  handled.  This  concept  implies  that  operators 
can  be  pooled  to  accomplish  more  tasking,  which  allows  for  the  possibility  of  the 
reduction  of  manpower. 
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Figure  13:  The  Perception  Engine 

The  perception  engine  works  exclusively  with  sensor  data,  and  not  fused  or 
combined  information.  The  idea  in  the  perception  engine  is  that  while  the  machine  is 
working  directly  with  the  data,  it  is  providing  the  man  with  advisory  information  at  a 
sensor/contact  level. 

The  human  operator  will  be  able  to  “drill”  down  to  see  the  raw  data  if  desired,  but 
otherwise  the  machines  are  creating  simply  pieces  of  sensor  dependent  “chunks”  of  data 
on  a  contact.  This  term  is  merely  being  used  conceptually  to  illustrate  data  flow,  and  still 
implies  no  particular  implementation. 

The  perception  engine  performing  these  computational  and  advisory  tasks  is  an 
enabler.  The  model  was  based  on  the  premise  that  machines  are  able  to  better  make 
recommendations  that  assist  an  operator  than  is  possible  today.  The  required  machine 
effectiveness  derived  from  the  model  may  be  used  in  the  future  as  input  requirements  for 
a  combat  system  developer. 
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a.  Contact  Processing/Contact  Assessment 


Contact  processing  and  assessment  were  decomposed  into  two  separate  tasks. 
This  allows  for  varying  levels  of  automation.  For  example,  the  contact  processing  can  be 
done  more  by  machine,  providing  suggestions  and  direction  to  a  human  operator.  The 
human  operator  acts  as  an  assessor  to  the  data  the  machine  has  derived  for  him.  This 
allows  more  flexibility  in  the  model  without  suggesting  implementation. 

2.  Comprehend 

The  Comprehension  Engine  consists  of  those  units  that  combine  the  sensor 
objects  into  a  situational  picture.  It  is  designed  to  aid  the  OOD  in  putting  together  his 
mental  model  of  the  current  situation. 

The  sensor  outputs  from  the  perception  engine  are  used  to  generate  trackers, 
perform  TMA  and  begin  fusing  those  outputs  together.  How  accurately  the  individual 
contacts  are  understood  (in  range  and  bearing)  allows  them  to  be  placed  in  the  world 
model  with  the  other  contacts  with  some  level  of  accuracy.  Once  the  contacts  are  placed 
in  the  world,  the  submarine  command  can  then  develop  the  understanding 
(comprehension)  of  the  situation  they  are  in.  Therefore,  the  accuracy  of  the  data  fusion 
implies  greater  probability  of  identification. 

The  Comprehension  Engine,  shown  in  Figure  14,  likely  has  some  sort  of  shared 
display  unit  in  which  the  command  team  of  the  crew  can  visualize  the  situation,  that  is, 
where  those  contacts  are  placed  in  the  world. 

The  team  considered  Air  Traffic  Control  as  a  model  for  monitoring  several 
contacts.  It  was  rejected  because  on  board  a  submarine  the  world  is  not  transparent,  as  it 
is  in  aviation.  Usually  the  contact  cannot  be  seen,  only  heard.  There  are  no  IFF  responses 
available  as  with  aircraft,  and  thus  location  is  only  a  guess.  Therefore,  the  SA  picture 
becomes  more  probabilistic  than  absolute.  This  train  of  thought  came  from  discussions  at 
the  Naval  War  College  [Bundy,  2010]. 
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Figure  14  the  Comprehension  Engine 
a.  TMA  and  Data  Fusion 

Although  the  sensors  themselves  are  outside  the  boundary  of  the  model,  a 
distribution  of  how  each  sensor  acts  was  derived.  These  curves  are  estimated  from 
interviews  with  SMEs  and  the  experience  of  the  team.  They  are  shown  in  Figure  15,  and 
represent  the  level  of  output  energy  of  the  contact  above  the  background  noise  (that  is,  a 
recognition  differential).  The  leftmost  vertical  (dashed)  line  is  the  cut-off  between 
detectable  and  not  detectable.  Nothing  to  the  left  is  detected.  Everything  to  the  right  is 
detected,  and  there  may  be  enough  energy  to  place  that  contact  in  a  higher  category 
(whether  or  not  the  contact  can  be  tracked,  classified  or  identified). 
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Figure  15  Sensor  Outputs  and  Fusion  /  TMA  outputs 

The  “fused”  location  (range  and  bearing)  certainty  is  the  maximum  accuracy  of 
both  range  and  bearing  of  all  the  sensors.  This  table  provides  confidence  estimates 
derived  by  the  team  with  inputs  derived  from  discussions  with  Mr.  Williamson 
[Williamson,  2010]  and  team  experience.  These  values  can  be  verified  and  modified  in 
the  future. 

The  current  fleet  guidance  is  to  perform  TMA  on  all  contacts.  In  heavy  contact 
areas,  this  is  not  always  possible  as  it  is  a  time  consuming  task,  and  must  be  continuously 
updated  depending  on  ownship  and  contact  motion.  Therefore,  the  crew  typically 
prioritizes  based  on  what  is  most  important  to  them  at  the  time.  This  is  governed  by  range 
(close  in  contacts  are  high  priority  to  avoid  an  accident)  and  type  (certain  contacts  are 
more  important,  such  as  warships).  The  concept  of  range  and  criticality  became  important 
to  the  model  as  the  man-machine  tradespace  was  developed. 
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Data  Fusion  is  the  process  of  combining  the  sensor  outputs  from  all  available 
sensors  into  a  single  target  vector  on  a  per  contact  basis.  Washburn’s  work  described  the 
TMA  process  of  analyzing  the  contact  localization  and  movement  estimation  based  on 
the  sensor  bearing  output  [Washburn,  2010].  The  system  goal  is  to  improve  the  accuracy 
of  the  bearing  measurements  in  an  attempt  to  reduce  the  error  ellipse  associated  with  the 
bearing  data.  Improved  sensor  accuracy,  or  the  use  of  more  sensors,  will  improve  the 
contact  bearing  estimates  which  will  improve  the  results  of  TMA. 
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Figure  16  Probabilistic  Data  Fusion 

As  with  the  sensor  outputs,  the  interest  was  not  so  much  in  the  actual  details,  but 
rather  the  confidence  of  how  well  the  fusion  was  performed.  To  validate  our 
methodology  of  deriving  the  fusion  and  TMA  output  confidence,  we  considered  the  set  of 
performance  measures  for  the  fusion  itself.  This  list  is  provided  by  James  Llinas  [Llinas, 
2009]: 


•  Detection  probability  -  probability  of  detecting  entities  as  a  function  of 
range,  signal-to-noise  ratio,  and  so  on 

•  False  alarm  rate  -  rate  at  which  noisy  or  spurious  signals  are  incorrectly 
identified  as  valid  targets 

•  Location  estimate  accuracy  -  the  accuracy  with  which  the  position  of  an 
entity  is  determined 
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•  Identification  probability  -  probability  of  correctly  identifying  an  entity  as 
a  target 

•  Identification  range  -  the  range  between  a  sensing  system  and  target  at 
which  the  probability  of  correct  identification  exceeds  an  established 
threshold 

•  Time  from  transmission  to  detect  -  time  delay  between  a  signal  emitted  by 
a  target  (or  by  an  active  sensor)  and  the  detection  by  a  fusion  system 

•  Target  classification  accuracy  -  ability  of  a  sensor  suite  and  fusion  system 
to  correctly  identify  a  target  as  a  member  of  a  general  (or  particular)  class 
or  category 

While  this  list  is  generated  for  the  data  fusion  itself,  this  study  is  at  a  higher  level 
for  some  of  these  measures.  That  is,  the  concern  is  with  the  confidence  of  the  output,  not 
the  output  itself.  They  are  not  used  directly,  but  rather  indirectly  in  the  form  of  validating 
the  contact  data. 

The  outcome  of  data  fusion  and  TMA  is  an  estimate  of  location  (and 
identification)  of  that  contact.  The  better  the  information  on  that  contact  implies  a  better 
location,  in  terms  of  range  and  bearing  from  the  submarine.  What  is  needed  for  the  model 
is  not  the  actual  location,  but  rather  the  confidence  of  the  location.  Therefore,  the  possible 
sensor  outcomes  (0-4)  are  correlated  to  a  location  confidence. 

The  location  estimate  accuracy,  classification  accuracy,  and  identification 
probability  were  included  in  the  probability  output  table.  This  location  estimate  includes 
two  parts:  bearing  and  range  of  the  contact  relative  to  ownship  position.  The  kinematics 
of  ownship-to-contact  relationship  is  accounted  for  in  the  variability  in  the  contact 
distribution  in  terms  of  differing  speeds,  aspects  and  depth. 

Although  time  is  included  in  the  model,  it  is  used  to  handle  the  lifespan  of  each 
contact  (how  long  they  are  within  detection  range).  Time  is  also  used  to  for  operator 
service  times  so  that  operator  workload  can  be  observed. 
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b.  Trackers 


Trackers  are  generally  used  more  in  the  area  of  perception  to  mark  where  the 
operator  should  be  looking.  In  this  project,  the  team  elected  to  locate  them  in  the 
comprehend  engine.  This  is  not  to  imply  implementation,  but  rather  to  allow  that 
sometime  in  the  future,  the  machine  may  be  able  to  track  across  sensor  type,  to  better 
comprehend  the  situation.  This  implies  a  higher  level  of  automation,  and  is  akin  to  data 
fusion,  which  belongs  in  the  comprehension  engine. 

3.  Projection 

The  last  part  of  the  SA  definition  is  "project"  [Endsley,  2006].  In  this  case,  that 
means  generating  appropriate  recommendations  for  action  based  on  the  current  situation. 
For  example,  if  a  contact  gets  too  close,  the  submarine  command  might  decide  to 
maneuver  around  it  or  fire  weapons.  Each  decision  has  risk  and  ramifications,  and  each 
depends  on  the  mission  requirements.  To  make  the  decision  to  maneuver,  for  example, 
requires  an  understanding  of  the  contacts  in  the  immediate  area  and  the  ability  to  project 
forward  in  time  to  predict  position  and  avoid  a  collision. 

There  are  many  possible  ways  of  handling  these  situational  projections.  Cues  can 
be  taken  from  Sheridan  in  his  levels  of  automation  as  shown  in  Table  5  [Sheridan,  2002], 
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Automation 

Level 

Description 

1 

Automation  offers  no  aid;  human  in  complete  control 

2 

Automation  suggests  multiple  alternatives;  filters  and  highlights  what  it 

considers  to  be  the  best  alternatives 

3 

Automation  selects  an  alternative;  one  set  of  information,  or  a  way  to  do 

the  task  and  suggests  it  to  the  person 

4 

Automation  carries  out  the  action  if  the  person  approves 

5 

Automation  provides  the  person  with  limited  time  to  veto  the  action 

before  it  carries  out  the  action. 

6 

Automation  carries  out  an  action  then  informs  the  person 

7 

Automation  carries  out  an  action  and  informs  the  person  only  if  asked 

8 

Automation  selects  method,  executes  task,  and  ignores  the  human  (i.e., 

the  human  has  no  veto  power  and  is  not  informed). 

Table  5  Automation  Level  Descriptions  [Sheridan,  2002] 


As  mentioned  above,  the  risk  is  too  great  to  leave  the  actual  performing  of  action 
to  a  machine  unless  a  high  level  of  trust  has  been  developed.  Developing  this  level  of 
trust  in  a  complex  system  is  discussed  in  an  article  by  Philip  Chapel  of  the  Australian 
Department  of  Defense,  in  which  a  system  can  be  used  to  alert  the  operator  to  the 
presence  of  mine-like  objects  and  other  significant  features  [Chapel,  2010]. 

“A  useful  rule-of-thumb  is  that,  for  an  automated 
detection  system  to  be  trusted,  the  expectation  of  detecting  a 
genuine  target  must  be  at  least  ten  times  the  expectation  of 
encountering  a  false  alarm” 

Chapel  goes  on  to  describe  two  scenarios  that  may  cause  the  system  not  to  be 
used.  First,  the  human  operators  outperform  the  machine,  making  the  machine  unreliable. 
Second,  the  operators  are  unreliable,  especially  when  the  detection  process  is  too 
difficult.  The  expectation  is  to  improve  machine  decision  making  processes  eventually, 
but  in  the  shorter  term,  it  might  not  be  possible.  To  some  degree,  the  surface  ship 
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community  has  been  able  to  do  this.  The  surface  sonar  community  is  more  aware  of  the 
environment  (for  example,  they  have  constant  weather,  Global  Positioning,  RADAR  and 
AIS  feedback).  They  do  not  have  the  disadvantage  of  relying  on  passive  sensors  and 
historical  databases  to  define  the  environment  as  does  the  submerged  submarine. 
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Figure  17  The  Projection  Engine 


Because  the  actual  action  to  be  taken  requires  understanding  of  the  particular 
mission  requirements,  it  was  decided  not  to  simulate  the  action.  Taking  correct  action 
depends  on  how  well  the  comprehension  is  developed.  It  is  assumed  that  the  best  possible 
situational  picture  will  enable  the  submarine  captain  to  make  the  correct  decision  (see 
Appendix  B).  Therefore,  in  the  model  developed,  the  actual  “act”  is  left  to  the  human. 
This  would  equate  to  (depending  on  the  action)  either  level  3  or  4  of  Sheridan’s 
automation  levels. 


Some  of  the  SA  development  (prediction)  can  be  automated.  Collisions  can  be 
projected  and  avoided,  as  well  as  warnings  of  increased  probability  of  ownship  detection. 
However,  much  cannot  be  automated.  For  example,  some  predictions  might  depend  on 
understanding  of  wartime  tactics.  Therefore,  the  machines  are  allowed  to  aid  in  the 
decision  making  process  (taking  on  an  advisory  role),  but  a  human,  who  can  not  only 
comprehend  the  current  situation,  but  also  make  decisions  based  on  his  needs,  must  make 
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the  ultimate  decision.  To  summarize,  “Accurate  choice  will  depend  on  good  SA,  but 
choice  is  not  the  same  as  SA  [Parasuraman,  et  al.,  2008]”.  Therefore,  this  is  left  as  a 
separate  step  in  the  model,  the  last  step  before  assessment  and  is  labeled  as  “Act”. 
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VII  MODELING  AND  SIMULATION 


A.  APPROACH 

As  described  previously  in  the  Systems  Engineering  process,  there  are  three 
models  developed  for  this  project;  architectural,  cost  and  ExtendSim®.  These  are 
described  in  the  following  sections. 

B.  MODEL  BLOCK  DIAGRAM 

The  block  diagram  of  the  model  is  illustrated  in  Figure  18.  It  is  based  on  the 
capabilities  shown  in  Figure  11.  The  following  sections  discuss  the  major  features  of  this 
model. 

1.  Contacts  and  Sensors 

The  actual  contact  details  and  characterization  are  outside  the  boundary  of  the 
model,  yet  they  are  required  to  perform  the  processing.  The  contacts  in  the  model  are: 

•  Warships, 

•  Submarines, 

•  Pleasure  craft, 

•  Merchants. 

The  modeled  sensor  outcomes  were  probabilistically  generalized,  as  it  is  beyond 
the  scope  of  this  study  to  directly  model  the  performance  of  the  sensors.  Therefore,  the 
sensors  in  the  model  output  the  probabilistic  outcomes  as  shown  in  Table  6.  In  the  real 
sensor  environment,  these  are  related  to  the  contact  SNR,  as  seen  at  the  sensor’s  receiver. 
These  outcomes  are  supplied  to  CCS,  where  further  processing  is  applied  to  support  the 
engagement  chain. 
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Arriving 
Contact  Input 


Sensor  Processsinq/Outputs 


•p.d.f. 

•Range, 

•Type 

•Time  to  Live 


PBB 

PNB 

ESED 

IMAGE 

Capability 

Capability 

Capability 

Capability 

Enabling 

Enabling 

Enabling 

Enabling 

Matrix 

Matrix 

Matrix 

Matrix 

Sensor  Outputs 
•Sensor 

•Strength 

•Bearing  (Estimate.  Conf) 
•Criticality  =f(Range,  Type) 


Figure  18  Model  Block  Diagram  with  Man-Machine  Tradespace  Highlighted 
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Outcome 

Category 

Description 

0 

No  Detect 

The  system  never  knew  the  contact  was  present 

1 

Detected 

The  system  knew  the  contact  was  there,  but  it  was 

not  clear  what  the  contact  was,  and  the  location  is 

not  clear. 

2 

Trackable 

The  system  found  the  contact,  and  was  able  to 

place  an  automated  tracker  on  it.  The  trackers  are 

not  perfect,  and  may  require  manual  updating  in 

some  circumstances  such  as  the  submarine  going 

through  a  turn. 

3 

Classifiable 

Not  only  does  the  system  know  where  it  is,  but  also 

there  is  enough  information  available  to  be  able  to 

tell  what  type  contact  it  is. 

4 

Identifiable 

Not  only  does  the  system  know  what  type  of 

contact  it  is,  but  also  it  knows  exactly  which 

vessel/hull  name  it  is.  This  implies  there  is  some 

known  information  on  that  vessel.  For  example,  the 

crew  can  see  via  periscope  imagery  that  the  sighted 

contact  is  an  aircraft  carrier  of  a  given  country. 

There  is  intelligence,  or  they  can  read  the  number 

off  the  front  of  the  contact,  that  it  is  that  exact 

aircraft  carrier. 

Table  6  Sensor  Outcomes 


2.  The  Man-Machine  Tradespace 

Figure  18  illustrates  the  tradespace  in  the  area  outlined  and  marked 
Man/Machine  Trade-space.  This  tradespace  consists  of: 

•  Percent  man  versus  machine  used  at  each  workstation, 

•  Number  of  men  available  to  perform  a  function, 


54 


•  Threshold  used  to  determine  if  a  contact  has  sufficient  priority  to 
engage  “man  as  operator”, 

•  Probability  that  the  man  will  accept  the  machine’s 
recommendation  or  a  more  in  depth  review  of  the 
recommendation. 

The  overall  effectiveness  of  the  system  is  driven  by  the  combined 
effectiveness  of  man  and  machine  over  time,  coupled  with  the  percent  man  or 
machine  at  each  workstation  supporting  a  particular  capability  of  the  engagement 
chain.  The  model  design  permits  modifying  the  balance  between  man  and 
machine  at  any  CCS  workstation.  In  the  model,  this  balance  is  maintained  by  the 
priority  threshold.  If  a  contact  priority  exceeds  the  priority  threshold,  that  contact 
is  presented  to  the  operator  with  amplifying  information,  track  history  and  track 
projection.  Otherwise,  the  contact  is  hidden  from  the  operator  except  as  a  process 
data  output.  If  the  operator  is  presented  with  the  contact,  the  operator  can  either 
accept  the  contact  or  decide  to  validate  the  contact.  The  priority  threshold  can  be 
thought  of  as  a  “machine  sensitivity  knob”,  which  could  easily  be  built  into  an 
implementation  of  CCS.  The  machine  sensitivity  knob  capability  represents  one 
of  the  major  new  architectural  features  of  the  proposed  construct.  One  of  the  key 
attributes  of  this  tradespace  detail  is  that,  as  trust  in  the  machine  as  operator 
(automation)  is  increased,  the  percent  man  can  be  decreased  (percent  machine 
increased). 

The  operational  parameter  that  controls  this  tradespace  is  Percent  Man  as 
indicated  by  the  green  box  in  the  top  center  of  Figure  18.  The  Percent  Machine  is 
calculated  as  (1-  Percent  Man).  The  Priority  Threshold  is  simply  set  to  the  Percent 
Machine  used  in  the  system. 

In  addition  to  the  machine  sensitivity  feature  presented  above,  the  second 
major  architectural  feature  of  this  construct  is  the  fusion  pipeline  shown  in  light 
blue  in  Figure  18.  This  pipeline  is  where  each  sensor  presents  its  contact  to  the 
CCS,  and  where  the  CCS  returns  fused  contact  information  to  the  sensors.  If  a 
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contact  exits  in  the  Fusion  Pipe  with  a  priority  lower  than  the  priority  threshold, 
then  it  is  simply  handled  by  the  machine  and  the  next  contact  is  processed. 

From  a  roles  and  responsibilities  perspective,  this  mechanism  assumes  that 
the  machine  is  acting  in  the  operator  role,  providing  advisory  information  to  the 
human.  If  the  human  rejects  the  advice,  then  he  is  forced  to  spend  more  time 
either  processing  it  himself  or  providing  guidance  to  the  machine.  The  human  can 
become  overloaded.  If  this  happens,  the  threshold  should  be  adjusted  such  that  the 
machine  is  handling  a  higher  percentage  of  the  contacts. 

3.  Contact  Prioritization 

One  important  part  of  the  sensor  outputs  is  a  contact  strength  (level  of 
detection),  which  influences  the  fusion  and  TMA.  The  blue  fusion  pipe  box  is 
already  described  above  in  the  Fusion  /  TMA  output  table.  “Criticality”  is  a  term 
used  to  identify  how  important  a  contact  was  to  the  model,  so  prioritization  of 
contacts  can  take  place.  For  simplicity,  the  team  agreed  that  ultimately  the 
criticality  of  each  contact  is  dependent  on  type:  submarine,  warship,  merchant  or 
pleasure  craft.  A  criticality  value  is  assigned  to  each  contact  type  based  on  how 
important  the  contact  is  to  the  safety  and  mission  of  the  submarine.  The  priority 
of  a  contact  is  calculated  as  a  function  of  the  normalized  range,  speed,  direction 
and  criticality  of  that  contact.  For  example,  a  distant  fishing  submarine  is  not  as 
critical  as  a  nearby  warship.  As  the  contact  approaches  ownship,  the  priority  and 
awareness  increase,  as  illustrated  in  Figure  19.  Regular  awareness  is  shown  in 
green,  elevated  awareness  is  shown  in  yellow,  high  awareness  is  shown  in  gray 
and  the  exclusion  zone  is  shown  in  orange. 
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Regular  Awareness 


Criticality  High 
Priority  High 


Criticality  High 
Priority  Medium 


Criticality  Low 
Priority  Low 


Figure  19  Contact  Prioritization  Illustration 

4.  Human  Effectiveness 

For  the  model,  it  was  assumed  that  the  processing  power  will  be  available 
to  process  as  many  contacts  as  are  needed.  It  is  also  assumed  that  the  costs  of 
additional  processing  power  will  be  negligible  in  the  future  based  on  the  historical 
trend  of  decreased  hardware  costs,  with  increased  performance.  Humans  on  the 
other  hand,  have  a  definite  processing  time,  which  is  defined  by  a  6-hour  shift 
cycle.  The  human  has  a  basic  service  time  that  is  adjusted  by  the  time  in  his  shift. 
Each  sensor  type  has  its  own  basic  processing  cycle  time. 

This  model  does  in  fact  change  the  operators’  CONOPS.  The  more 
machine  utilization  (lowering  the  %  Man),  the  human  becomes  a  “supervisor” 
whose  job  it  is  to  accept  or  reject  the  machine’s  recommendations. 

This,  in  effect,  changes  the  role  of  the  human  in  the  system.  The 
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automation  should  be  better  at  low  level  calculations  and  repetitive  tasks  (the 
menial  and  error-prone  parts).  Therefore,  it  is  assumed  that  the  automation  would 
start  there,  and  as  it  becomes  better  could  take  over  more  of  the  human  performed 
low-level  jobs. 

5.  Pooled  Resources 

Another  key  aspect  of  the  model  is  the  concept  of  pooled  resources. 
Rather  than  having  an  individual  operator  for  each  sensor,  the  model  relies  on  a 
pool  of  qualified  operators  that  can  be  assigned  to  various  process  tasks.  This 
"pool"  allows  for  maximum  utilization  of  fewer  operators  to  perform  the  same 
function  as  many  specialized  operators.  In  the  model,  there  are  two  resource 
pools,  one  serving  the  TMA  function  and  the  other  performing  detect  and  identify 
functions  for  all  sensors. 

6.  Human  System  Interface 

It  is  also  assumed  that  the  human  interface  is  advanced  enough  such  that 
all  the  information  the  operator  requires  to  make  a  good  decision  is  organized  in  a 
way  that  it  is  readily  available. 

Human  System  Integration  (HSI)  is  accounted  for  in  the  model.  The 
Contact  Service  time  per  Sensor  portion  (right  side,  in  the  middle  of  Figure  18) 
considers  this,  and  is  adjusted.  It  could  be  said  that  HSI  actually  increases  or 
decreases  human  effectiveness  (upper  right  corner  of  Figure  18).  Also,  this 
mechanism  was  used  to  generate  a  varying  human  effectiveness  based  on 
circadian  rhythms  in 

Figure  20,  also  supported  by  the  model  [Duplessis  et.  al,  2007]. 
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Human  Effectiveness 


Man  Efficiency 


On-Station  Time  (Hrs) 


Figure  20  Human  Effectiveness 

7.  Operator  Load  and  CONEMP 

The  level  of  machine  utilization  and  effectiveness  was  allowed  to  vary, 
thereby  changing  the  model  outcomes.  The  more  automation  was  utilized  and  the 
higher  the  effectiveness  of  that  automation  resulted  in  outcomes  greater  or  less 
than  that  of  a  near  wholly  human  operated  submarine  combat  system. 

One  important  aspect  not  considered  in  the  model  is  operator 
complacency.  Since  the  operator  is  now  required  to  perform  more  (albeit  a  higher 
level)  jobs,  and  if  he  accepts  most  of  the  machine  recommendations  without 
checking  the  data,  he  may  lose  effectiveness.  The  hope  is  that  the  higher  level  of 
data  and  indeed  responsibility  of  the  operator  should  keep  him  closer  to  the 
optimal  range  of  attention  (arousal).  “Moray  and  Inagaki  (2000)  suggested  that 
the  attention  allocation  strategy  could  be  rational  and,  furthermore,  that 
complacency  should  be  inferred  only  if  the  rate  of  monitoring  was  below  that  of 
an  “optimal”  observer  who  was  required  to  attend  to  many  sources  of  information 
(the  automated  task  being  one  such  source)”  [Parasuraman,  et  al.,  2008], 
Therefore,  the  idea  of  giving  him  more  tasks  is  not  just  for  purposes  of  lowering 
manpower,  but  rather  it  is  required  in  an  alternate  CONEMP  in  order  to  maintain 
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vigilance.  “Moray  and  Inagaki  suggested  that  a  human  operator  who  monitored 
automation  at  a  lesser  rate  than  the  optimal  Nyquist  frequency  was  complacent 
and  the  one  who  monitored  at  a  greater  rate  was  skeptical,  whereas  the  one  who 
monitored  at  the  optimal  rate  was  eutectic  (or  well  calibrated)”  [Parasuraman,  et 
ah,  2008], 

This  concept  is  reinforced  by  the  Yerkes-Dodson  law,  in  which  it  is  seen 
that  if  the  operator  is  not  engaged  enough,  his  level  of  performance  is  lower,  as 
well  as  if  he  is  too  engaged.  A  variation  of  the  Yerkes-Dodson  law  is  depicted  in 
Figure  21. 


Figure  21  Arousal  Level  [Schmidt,  2010] 

The  operator  will  begin  to  shed  tasks  in  times  of  heavier  stress  (more 
contacts,  mission  area,  etc.).  This  is  the  time  where  it  is  more  important  than  ever 
to  have  automation  (that  the  operator  can  trust)  in  the  system  to  assist  him. 

Too  much  trust  in  the  automation  implies  an  increase  in  complacency.  The 
human  becomes  reliant  on  the  machine,  and  essentially  bored.  Therefore,  in  order 
to  maintain  vigilance  (lower  complacency),  the  answer  is  not  to  simply  lower  the 
workload.  Man  must  be  kept  “in  the  loop”,  that  is,  engaged.  In  order  to 
accomplish  this,  more  reliance  on  automation  is  required,  but  also  the  human  must 
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have  more  tasks  to  accomplish.  The  operator  is  no  longer  bored  by  his  job,  but 
rather  can  accomplish  more  when  aided  by  automation.  This  has  the  effect  of 
raising  the  operator  to  the  supervisor  level,  and  results  in  his  processing  machine 
advisories  from  several  different  sources. 

Verification  of  combined  machine  outputs  is  not  validated  by  looking  at 
the  raw  data,  but  rather  confirmed  or  denied  by  use  of  other  systems.  For 
example,  if  a  contact  is  detected  by  a  system,  the  system  automatically  initializes 
trackers  and  begins  the  process  of  classification  and  identification.  Today,  the 
human  checks  the  inputs  to  this  automation.  The  human  is  now  forced  to 
reconstruct  the  same  process  the  automation  has  already  completed.  Rather,  if  two 
separate  systems  can  come  to  a  similar  conclusion,  then  the  outputs  are  reinforced 
without  the  need  for  a  human  to  examine  the  raw  data.  As  an  example,  the  sonar 
operator  is  elevated  from  looking  specifically  at  the  towed  array  data  to  looking  at 
the  automated  outputs  from  all  the  sensors  combined  and  verified  with  multiple 
automated  methods.  This  effectively  eliminates  operators  of  other  arrays. 

C.  COST  MODEL 

The  following  assumptions  were  made  pertaining  to  cost  data  in  support  of 
the  USN's  wide  initiative  to  reduce  TOC: 

•  Applicable  only  to  Submarine  personnel, 

•  Addresses  Combat  Control  operations, 

•  Does  not  incorporate  effects  due  to  personnel  reduction  outside  combat 

control, 

o  Crew’s  mess 
o  Damage  Control 
o  General  ship  maintenance 

•  No  consequence  will  be  incurred  by  reducing  personnel. 

The  thought  process  is  that  if  manning  can  be  reduced  on  a  platform  as  a 

result  of  applying  automation  to  human  intensive  processes,  then  actual  cost 
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reduction  can  be  realized  by  the  annual  labor  rate  associated  with  the  elimination 
of  the  manual  tasks.  Aside  from  the  reduction  of  actual  labor  cost  by  replacing  a 
human  operated  task  with  a  machine  automated  task,  RTOC  will  be  seen  at  the 
operating  level  of  the  platform.  This  includes,  but  is  not  limited  to,  the  amount  of 
provisions  (food),  berthing,  linen  or  the  basic  necessitates  needed  per  sailor. 
Indirect  areas  that  might  contribute  to  RTOC  based  on  reduced  manning  include 
the  amount  of  oxygen  the  air  handling  system  will  need  to  generate,  the  amount  of 
water  purified  and  the  amount  of  waste  produced.  On  an  even  more  remote  level, 
the  administration  of  sailors,  such  as  accountability,  medical  care  or  liability 
(annual  or  sick  leave)  could  certainly  contribute  to  lowering  the  TOC  of  operation 
of  a  submarine. 

Total  Ownership  Cost  (TOC)  is  defined  per  platform  as  in  Allison 
[Allison,  2000]: 


(6) 

Direct  Unit  Cost  (DUC)  and  Indirect  Cost  (IC)  are  where  the  realization  or 
reductions  can  be  seen.  The  Direct  Intermediate  Maintenance  Cost  (DIC)  and 
Direct  Depot  Maintenance  Cost  (DDC)  are  assumed  not  affected  by  the  analysis 
put  forth  from  SCSEP  and  are  therefore  outside  the  scope  of  this  project.  To 
reduce  TOC  for  the  future  VA  fleet  (and  others)  with  respect  to  the  Combat 
System,  equation  (6)  can  be  modified  to: 


(7) 


Where: 

RTOCva  -  VA  Fleet  RTOC  (projected  at  18  platforms), 
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Future  VA  Fleet202o  -  Quantity  of  VA  submarines  projected  in  2020, 
RTOCcost  Savings  per  Combat  System  -  Cost  Savings  per  Combat  System. 

Table  7  shows  the  number  if  enlisted  personnel  on  board  a  VA  Class 
submarine.  These  personnel  and  rates  were  selected  to  show  the  impact  of 
reduced  manning  for  one  submarine.  The  enlisted  personnel  were  determined  by 
analysis  of  the  Automated  Readiness  Information  System  (ARIS)  Rating  Control 
Numbers  (RCNs)  for  the  USS  New  Hampshire  [Devers,  2011]. 


Enlisted  Combat  System  Personnel 

Sonar 

Technician 

Submarine 

(STS) 

Fire  Control 
Technicians 
(FT) 

Torpedo  (TM) 

Navigation/ 

Communication 

Electronics 

Technicians 

(ET) 

Radioman 

Senior 

8 

3 

3 

4 

3 

Junior 

9 

4 

4 

5 

5 

17 

7 

7 

9 

8 

Table  7  Summary  of  USS  New  Hampshire  Enlisted  Personnel 


By  coupling  the  data  shown  in  Table  7  and  the  results  that  are  output  by 
the  SCSEP  model  (see  section  VII),  the  number  of  sailors  reduced  on  a  VA 
platform  can  be  quantified  into  dollars. 

Table  8  is  derived  from  raw  VAMOSC-ISR  Data,  converted  using  current 
inflation  indices  and  illustrates  the  overall  cost  for  enlisted  sailors  projected  for 
Fiscal  Years  (FY)  201 1  thru  2020.  [VAMOSC  2010]  The  ten  year  projected  cost 
per  person  per  platform  is  used  to  illustrate  the  overall  magnitude  of  the  cost 
reduction.  For  example,  if  the  reduction  is  in  terms  of  4  sailors,  over  the  course  of 
anticipated  ten  years  the  savings  would  be  $4,141,749.  Projecting  this  platform 
savings  to  the  fleet  of  eighteen  (18)  platforms  would  amount  to  a  RTOC  of 
$74,551,489. 


63 


Number  of  Enlisted  Sailors 


FY 

Index 

1 

2 

3 

4 

5 

6 

7 

25 

2011 

1.0000 

$92,757 

$185,515 

$278,272 

$371,029 

$463,787 

$556,544 

$649,301 

$2,318,925 

2012 

1.0155 

$94,195 

$188,390 

$282,586 

$376,781 

$470,976 

$565,171 

$659,367 

$2,354,880 

2013 

1.0371 

$96,199 

$192,398 

$288,596 

$384,795 

$480,994 

$577,193 

$673,392 

$2,404,970 

2014 

1.0649 

$98,774 

$197,548 

$296,323 

$395,097 

$493,871 

$592,645 

$691,419 

$2,469,354 

2015 

1.0947 

$101,540 

$203,080 

$304,620 

$406,159 

$507,699 

$609,239 

$710,779 

$2,538,496 

2016 

1.1253 

$104,383 

$208,766 

$313,149 

$417,532 

$521,915 

$626,298 

$730,681 

$2,609,574 

2017 

1.1568 

$107,306 

$214,611 

$321,917 

$429,223 

$536,528 

$643,834 

$751,140 

$2,682,642 

2018 

1.1892 

$110,310 

$220,621 

$330,931 

$441,241 

$551,551 

$661,862 

$772,172 

$2,757,756 

2019 

1.2225 

$113,399 

$226,798 

$340,197 

$453,596 

$566,995 

$680,394 

$793,793 

$2,834,973 

2020 

1.2568 

$116,574 

$233,148 

$349,722 

$466,296 

$582,871 

$699,445 

$816,019 

$2,914,353 

Totals 

$1,035,437 

'$2,070,875 

'$3,106,312 

'$4,141,749 

r$5,177,187 

'$6,212,624 

'$7,248,061 

$25,885,926 

Table  8  RTOC  of  Personnel  Labor  Summary 


VA  DUC  Savings  Per  Enlisted  Sailor 


2011  2012  2013  2014  2015  2016  2017  2018  2019  2020 


Figure  22:  VA  DUC  Savings  Per  Enlisted  Sailor  Reduction 

Figure  22  uses  the  cost  data  in  Table  8  to  show  the  overall  potential  RTOC 
for  VA  Class  over  a  ten  year  period.  This  savings  only  applies  to  personnel  labor 
costs.  The  net  cost  reduction  of  the  entire  platform  is  out  of  scope  of  this  project. 
However,  numerous  studies  have  been  conducted  that  have  addressed  the  net 
reduction  in  manning.  One  such  study  specifically  focused  on  cost  model  for 
United  States  Nuclear  Submarines  [Allison,  2000].  Coupling  this  with  other 
studies  and  this  project's  results  could  be  used  to  spawn  a  study  on  net  RTOC  cost 
figures. 
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D.  EXTENDSIM®  MODEL  DETAILS 

1.  Approach 

A  discrete-event  simulation  based  on  the  architectural  model  shown  in 
Figure  18,  above,  was  developed  using  ExtendSim7®.  The  design  of  the  model, 
as  well  as  key  assumptions  and  parameters,  are  discussed  in  the  following 
paragraphs. 

The  simulations  were  based  on  the  following  concepts  and  derived  from 
multiple  Design  of  Experiment  (DOE)  sources: 

•  Input  factors:  These  are  also  called  independent  variables.  These 
are  the  inputs  that  the  designer  has  control  over. 

•  Noise  factors:  These  are  sources  of  uncontrolled  variations  in  a 
process.  These  have  been  modeled  as  probability  functions  or  in 
one  case  a  periodic  function  of  time. 

•  Output  factors:  These  are  also  called  dependent  variables.  For  any 
process,  there  can  be  many  output  factors.  It  is  best  practice  to  only 
be  concerned  with  those  that  relate  to  the  outcome  of  the  process 
that  are  system  KPPs. 

For  each  sensor  in  the  contact  processing  there  are  three  input  factors  that 
are  modeled  as  operational  parameters:  (a)  percent  man  involvement,  (b)  the 
number  of  operators  available,  and  (c)  the  effectiveness  of  the  machine  relative  to 
man’s  top  effectiveness. 

In  addition  to  these  input  factors,  there  are  five  noise  factors  in  the  basic 
model.  These  noise  factors  include:  (a)  the  Poisson  arrival  rate  for  each  type  of 
contact,  (b)  life  time  for  each  arriving  contact,  (c)  the  signal  level  for  each 
arriving  contact,  (d)  the  exponentially  distributed  service  time  for  man  as  backup 
operator  or  supervisor  for  each  contact,  and  (e)  human  effectiveness  at  any  given 
time.  It  is  assumed  that  once  each  of  the  sensors  detects  a  given  contact,  its  signal 
level  is  constant.  Holding  the  signal  level  constant  was  done  for  simplicity.  The 
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overall  effect  of  signal  fading  is  incorporated  into  the  probability  distributions 
related  to  sensor  capabilities  and  the  TMA  and  Accuracy  measures. 

Table  9  shows  a  complete  list  of  the  input  and  noise  factors  that  were 
implemented  to  support  the  simulation  runs. 

Unless  otherwise  specified,  the  values  used  for  the  model  parameters 
described  below  were  all  assumed  based  on  the  best  judgment  of  project 
members’  in  our  respective  discipline  and  engineering  judgments. 


Input  Factors 


%  Man  BBG  Process 
%  Man  NBG  Process 
%  Man  Visual  Process 
%  Man  ES  Process 
%  Man  Update  Tracker  Process 
%  Man  TMA  Process 

#  Men  TMA 

#  Men  Pool 


Machine  Effectiveness 


Noise  Factors 

Submarine 

Poisson  arrival  rate 

Warship 

Poisson  arrival  rate 

Merchant 

Poisson  arrival  rate 

Pleasure  Craft 

Poisson  arrival  rate 

Contact  life  time 

Uniform  distribution 

Human  effectiveness 

Periodic  function  of  time 

Human  contact  service  time 

Exponential  distribution 

Contact  signal  strength 

Based  on  zero  mean  normal 

distributions,  and  a  function  of  sensor 

Table  9  ExtendSim®  Input  and  Noise  Factors 
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The  output  factors  measured  are  the  four  simulation  related  CCS  KPPs 
discussed  above: 

•  Percent  maintained  contacts, 

•  Fusion  score, 

•  Accuracy  measure,  and 

•  Number  of  men  used. 

2.  Contacts  Attributes. 

In  the  model,  contacts  are  generated  randomly,  and  then  each  contact  is 
passed  to  a  detection  and  identification  process  for  NBG,  BBG,  Imagery,  ES  and 
AIS  sensors.  The  contacts  are  then  assigned  a  status  of  non-detected,  detected, 
trackable,  classifiable  or  identifiable.  After  being  processed  for  detection  and 
identification  the  contacts  get  fused,  their  trackers  are  maintained  continuously, 
for  the  lifespan  of  the  contact  and  finally  TMA  is  performed.  At  each  process  in 
the  model,  data  is  recorded  and  sent  to  an  output  Excel  file  for  post  analysis. 

The  four  contact  types  (submarines,  warships,  merchant  ships,  and 
pleasure  craft)  are  generated  via  a  Poisson  distribution.  The  inter-arrival  times  that 
were  selected  were  based  on  a  worst  case  surface  transit  through  the  Straits  of 
Gibraltar.  Calculation  of  the  contact  density  over  a  period  of  several  hours  was 
performed  using  VT  Explorer®  AIS  software.  The  counting  included  the  binning 
of  the  contacts  into  the  categories  of  warship,  merchant  and  pleasure  craft.  The 
mean  time  between  arrivals  is  shown  below  for  each  contact  type. 


Mean  Interarrival  Times  (minutes) 

Submarine 

Warship 

Merchant 

Pleasure 

60 

60 

5.71 

36 

Table  10  Contact  Arrival  Rates 


Once  contacts  are  generated,  they  are  assigned  various  attributes,  which 
are  utilized  throughout  the  model.  Two  of  the  attributes  are  visual  and  acoustic 
strength.  These  attributes  are  compared  against  given  thresholds  to  determine  the 
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percentage  of  contacts  that  are  non-detectable,  detectable,  trackable,  identifiable 
or  classifiable.  The  thresholds  were  based  on  achieving  the  probability 
distributions  for  each  sensor  type  shown  in  Table  11.  These  probability 
distribution  functions  are  derived  from  SME  interviews  and  team  experience.  The 
visual  and  acoustic  strength  are  later  multiplied  by  the  man  and/or  machine 
effectiveness,  which  results  in  a  shift  of  those  probability  distributions.  The  man’s 
effectiveness  from 

Figure  20  varies  with  time  from  a  minimum  of  0.69  to  a  max  of  1.0  with 
an  average  of  0.81  [Duplessis,  et  al.,  2007],  As  man’s  effectiveness  decreases,  he 
makes  more  mistakes  and  takes  more  time  to  accomplish  tasks.  The  machine 
effectiveness  was  varied  as  one  of  the  input  variables  with  a  minimum  of  0.345 
and  a  max  of  2,  which  is  twice  the  range  of  the  man’s  effectiveness. 

Another  attribute  that  is  assigned  to  each  contact  is  lifespan.  The  lifespan 
is  intended  to  simulate  the  amount  of  time  that  each  contact  is  within  sensor 
range.  The  lifespan  for  all  contacts  was  assumed  to  be  a  uniform  distribution 
between  10  and  60  minutes.  If  contacts  were  not  detected  before  they  reached 
their  lifespan,  they  were  assigned  a  status  of  non-detected  by  each  sensor.  The 
lifespan  was  also  utilized  during  the  update  tracker  portion  of  the  model.  Contacts 
were  cycled  through  the  update  tracker  process  continuously  until  their  lifespan 
was  reached. 


Probability  Distributions  (%) 

Non- 

Detectable 

Detectable 

Trackable 

Classifiable 

Identifiable 

BBG 

10 

30 

60 

NBG 

10 

30 

20 

40 

ES 

20 

30 

50 

Imaging 

10 

50 

20 

20 

AIS  Merchants 

10 

90 

AIS  Warships 

90 

10 

AIS  Sub/Pleasure 

100 

0 

Table  11  Sensor  Probability  Distributions 
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One  feature  included  to  model  the  effects  of  an  overloaded  operator  is  a 
measure  of  how  many  contacts  wait  an  excessive  amount  of  time  before  they  are 
processed  by  a  person.  Prior  to  each  detection  process,  the  model  utilizes  a  queue 
to  simulate  contacts  that  have  not  been  detected  because  they  are  waiting  for  the 
operator  to  have  time  to  process  them.  For  the  NBG,  BBG,  and  imagery  detection 
processes,  if  contacts  stayed  in  that  queue  for  greater  than  5  minutes,  they  were 
declared  stale  targets  and  assigned  a  non-detected  status. 

3.  Human  Effectiveness 

Another  key  factor  used  by  the  model  is  the  basic  service  time  needed  to 
perform  each  task.  Table  12  shows  the  service  times  derived  from  SME 
interviews  and  team  experience  that  were  utilized  in  the  simulation.  The  manual 
service  time  is  assumed  to  be  the  time  it  takes  a  man  to  perform  a  task  without 
assistance  from  a  machine.  As  discussed  in  the  modeling  and  simulation  approach 
section,  when  the  percent  machine  is  increased,  the  man’s  role  becomes  more  of 
an  advisory  role  and  therefore  the  basic  service  times  utilized  for  that  process 
were  assumed  to  be  20%  of  the  values  shown  in  Table  12. 


Basic  Manual  Service  Time  (seconds) 

Non- 

Detectable 

Detectable 

Trackable 

Classifiable 

Identifiable 

BBG 

0 

30 

67.5 

NBG 

0 

400 

500 

255 

ES 

0 

4 

60 

Imaging 

0 

10 

12 

63 

Update  Trackers 

0 

30 

30 

30 

TMA 

0 

300 

300 

300 

Table  12  Basic  Service  Times 


4.  Contact  Prioritization 

Another  attribute  assigned  to  each  contact  is  level  of  criticality.  The  level 
of  criticality  is  based  on  the  type  of  contact.  Submarines  and  warships  were 
assigned  a  value  of  1,  merchant  ships  were  assigned  a  value  of  0.5  and  pleasure 
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crafts  were  assigned  a  value  of  0.1.  The  criticality  was  then  multiplied  by  a  range 
factor  to  determine  an  overall  contact  priority.  The  result  is  that  closer  contacts 
with  higher  criticality  level  are  assigned  a  high  level  of  priority.  Likewise  the 
lower  the  criticality  level  and  the  farther  away  the  contact  is,  the  lower  its  priority 
level. 


5.  Contact  Fusion 

The  model  simulates  contact  fusion  by  comparing  the  status  of  each 
contact  after  they  have  been  processed  by  their  respective  sensors  in  accordance 
with  the  values  shown  in  Table  13.  Sensor  contribution  is  based  on  how  much  a 
given  sensor,  independent  of  other  factors,  contributes  to  fusion.  The  capability 
contribution  shows  the  strength  of  the  fusion  contribution,  with  tracking 
contributing  90%  of  the  fusion,  tracking  and  classification  contributing  95%,  and 
tracking,  classification,  and  identification  contributing  100%.  For  example,  a 
contact  that  is  identifiable  by  the  NBG  sensor  would  be  assigned  an  NBG  value  of 
0.4.  These  scores  are  then  used  to  calculate  a  fusion  score.  The  maximum  sensor 
fusion  values  for  each  sensor  are  added  together  and  divided  by  the  maximum 
possible  fusion  score  (0.955)  to  arrive  at  the  fusion  score  for  each  contact. 


Capability  Contribution 

Sensor 

Track 

Classify 

Identify 

Sensor 

Contribution 


0.9 


0.95 


1 


Table  13  Master  Fusion  Table  with  all  5  Sensors 


6.  Contact  TMA 

TMA  is  handled  in  the  model  by  reading  the  status  of  each  contact  after  it 
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Maximum  Score 


has  been  processed  and  by  utilizing  the  confidence  values  in  Table  14.  Each 
contact  is  sequentially  assigned  target  range  and  bearing  confidence  values,  which 
vary  from  0  to  1  depending  on  how  well  the  targets  range  and  bearing  are  known. 
The  target  range  confidence  is  calculated  by  finding  the  maximum  confidence 
value  from  each  sensor  for  a  given  contact. 


Sensor 

Possible 

Outcomes 

Target  Range 
Confidence 

Target  Bearing 
Confidence 

Non-Detectable 

0 

0 

BBG 

Detectable 

0 

0.8 

Trackable 

0.5 

0.9 

Non-Detectable 

0 

0 

NBG 

Detectable 

0 

0.8 

Trackable 

0.5 

0.9 

Classifiable 

0.5 

1 

Non-Detectable 

0 

0 

ES 

Detectable 

0 

0.2 

Classifiable 

0.6 

0.8 

Non-Detectable 

0 

0 

Imagery 

Trackable 

0.6 

0.9 

Classifiable 

0.9 

1 

Identifiable 

0.9 

1 

AIS 

Non-Detectable 

0 

0 

Identifiable 

1 

1 

Table  14  TMA  Confidence  Values 


The  target  bearing  confidence  is  calculated  by  taking  an  average  of  the 
confidence  values  from  each  of  the  5  sensors. 

7.  Man/Machine  Tradespace 

Every  contact  is  initially  processed  by  a  machine  and  then  there  are  three 
possible  follow  on  actions.  The  first  action  is  that  low  priority  contacts  are  simply 
passed  on  and  not  looked  at  by  a  person.  The  second  action  is  for  some  portion  of 
the  higher  priority  contacts  to  be  reviewed  by  a  person  in  a  supervisory  role  where 
he  has  the  option  to  agree  with  the  machines  assessment  of  that  contact.  The  final 
action  is  that  the  man  reviews  the  other  two  choices  are  to  be  reviewed  by  a 
person  in  a  supervisory  role  or  to  completely  reprocess  the  contact  manually. 
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8.  Triggers 

Triggers  in  the  model  kick  off  certain  processing  based  on  Figure  15. 
Since  this  model  is  probabilistic,  these  triggers  act  like  thresholds  and  once 
exceeded  the  processing  of  the  contact  is  initialized.  Once  the  contact  arrival 
trigger  was  reached  the  contact  was  moved  to  the  contact  detection  function.  Once 
the  contact  detection  trigger  threshold  was  exceeded  the  contact  was  moved 
forward  to  the  identify  function.  If  at  any  point  the  contact  was  not  generating  a 
threshold  level  it  would  reach  an  end  of  life  trigger  and  be  considered  a  stale 
contact. 


9.  Simulation  Weaknesses 

There  are  two  weaknesses  in  the  simulation  with  respect  to  those  scenarios 
in  which  the  submarine  is  not  always  at  PD.  The  first  is  that  the  priority  of 
contacts  is  not  adjusted  once  the  submarine  submerges,  which  may  not  be  a 
reasonable  assumption.  While  surfaced,  all  contacts  are  a  concern  and  become  a 
high  priority  at  close  range.  Once  submerged  the  model  handles  non-military 
contacts  as  low  priority. 

The  second  weakness  in  the  model  is  how  the  master  fusion  table  is 
calculated.  As  described  above,  the  contact  fusion  score  is  adjusted  by  dividing 
the  fusion  score  by  the  maximum  value  achievable,  the  sum  of  all  five  sensors. 
When  at-depth,  there  are  only  two  sensors  available.  When  not  at  PD,  this  was 
accounted  for  in  post-processing  of  the  output  data,  by  dividing  the  fusion  score 
output  value  by  the  adjusted  maximum  contribution  of  the  BBG  and  NBG  sensors 
and  the  percent  time  of  the  submarine  at-depth.  The  maximum  sensor  fusion 
values  for  each  sensor  are  added  together  and  divided  by  the  max  possible  fusion 
score  (0.56)  to  arrive  at  the  fusion  score  for  each  contact.  Table  15  summarizes 
the  calculations  for  two  sensors. 
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Capability  Contribution 

T 

C 

%  Sensor  Contribution 

NBG 

0.4 

BBG 

0.2 

0.9 


0.36 


0.18 


Table  15  Master  Fusion  Table  with  2  Sensors 


E.  DETERMINING  SYSTEM  PARAMETER  VALUES 


Using  DOE  methods,  as  implemented  in  the  JMP9®  software  package, 
and  as  described  in  Appendix  C,  the  number  of  runs  and  the  output  values  were 
determined.  For  this  project,  the  DOE  was  set  up  for  nine  input  factors,  with  both 
second  order  and  two-way  interaction  effects.  This  resulted  in  64  runs,  as  shown 
in  Appendix  C. 

The  following  are  the  operational  parameters  that  represent  the  tradespace 
discussed  above:  percent  man  for  BBG,  NBG,  Visual  Process,  ES,  Update 
Tracker,  and  TMA  processes.  Other  operational  parameters  are  the  number  of 
men  available  to  conduct  TMA  and  run  the  sensor  pool.  Machine  effectiveness 
versus  human  effectiveness  also  contributes  to  the  tradespace  parameters.  One 
method  to  determine  the  optimum  values  is  to  run  a  series  of  experiments,  with 
the  input  values  determined  by  the  DOE  methods  [Yang,  2009].  The  JMP9® 
software  analysis  package  provides  a  custom  DOE  module  that  allows  a  fractional 
factorial  DOE  screening  that  identifies  those  input  factors  to  which  the  system  is 
either  sensitive  or  insensitive.  In  addition,  the  software  was  used  to  determine 
sensitivities  to  interactions  between  input  factors  and  powers  of  the  input  factors. 
For  this  project,  two  way  interactions  and  second  order  powers  were  utilized.  In 
order  to  perform  a  DOE,  the  range  over  which  each  parameter  had  to  be  evaluated 
was  specified.  Table  16  below  shows  the  range  for  each,  with  rationale  for  range 
selection. 
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Input  Parameter 

Low 

value 

High 

Value 

Range  Rationale 

%  Man  BBG  Process 

25 

75 

High  value:  The  current  implementation  for 
each  is  already  somewhat  automated,  with  an 
estimate  of  25%  machine  contribution  (75% 
man). 

Low  Value:  The  basic  assumptions  on  the 

%  Man  NBG  Process 

25 

75 

%  Man  Visual  Process 

25 

75 

%  Man  ES  Process 

25 

75 

machine  effectiveness  and  speed  of  machine 
based  decisions,  vice  man,  would  typically 
drive  these  parameters  to  the  low  end; 
however,  this  is  limited  to  25%  as  it  was  felt 
that  full  automation  of  a  submarine  would  be 
too  risky. 

%  Man  Update  Tracker 
Process 

25 

75 

%  Man  TMA  Process 

25 

75 

Number  of  Men  TMA 
Process 

1 

4 

High  Value:  The  number  of  operators  per 
shift  currently  assigned  to  the  task 

Low  Value:  The  minimum  based  on  1 

Number  of  Men  Sensor 

Pool 

3 

6 

operator  per  sonar  sensor,  plus  sharing  EW, 
one  for  TMA,  and  one  for  TMA 

Machine  Effectiveness 
relative  to  man 

0.345 

2 

High  value:  Based  on  twice  the  maximum 
man  effectiveness  used  in  the  model. 

Low  Value:  Based  on  1/2  of  the  minimum 
man  effectiveness  used  in  the  model. 

Table  16  Combinational  DOE  inputs 
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VIII  MODELING  RESULTS 


A.  OPERATIONAL  PARAMETERS 


Table  17  shows  the  optimal  values  for  the  CCS  operational  parameters  resulting  from 
JMP9®’s  Optimization  analysis  of  the  64  runs  defined  by  JMP9®'s  DOE  module,  as  described 
above. 


%  Machine 

Number  of 
Operators 

Machine 

Effectiveness 

BBG 

Process 

NBG 

Process 

Visual 

Process 

ES 

Process 

Update 

Tracker 

Process 

TMA 

Process 

TMA 

Sensor 

Pool 

75 

75 

75 

25 

75 

75 

1 

3 

2 

Table  17  CCS  Optimized  Operational  Parameters 


B.  KEY  PERFORMANCE  PARAMETERS 


The  KPPs  and  their  weights  for  the  architectural  construct  are  listed  in  Table  18.  The 
operational  values  from  Table  17  and  Table  18  were  then  used  in  the  ExtendSim®  model  and  a 
simulation  was  run  to  determine  the  resulting  KPP  average  values. 

Table  18  shows  the  resulting  KPP  values  and  scores  with  the  corresponding  CCS  score. 
Also  shown  are  the  results  for  the  same  operational  parameters,  with  the  machine  effectiveness 
set  to  0.5  and  1,  relative  to  man  effectiveness. 
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Simulated  CCS  KPPs 

Percent 

Maintained 

Contacts 

Fusion  Score 
of 

Contributin 

Accuracy 

Measure 

Average 
number 
of  men 

g  Sensors 

used 

KPP  Bounds 

Scoring  weight 

0.3191 

0.0383 

0.0960 

0.5470 

Scoring 

Run 

Threshold 

0.8 

0.7 

0.5 

8 

Score 

Goal 

0.999 

0.95 

0.9 

3 

Optimized 

Values 

1.000 

3.480 

0.839 

3.193 

0.965 

Resultant  Scores 

1.000 

1.000 

0.848 

0.961 

Min  Machine 

Average  Values 

0.9974 

0.8496 

0.7419 

3.1888 

0.924 

Resultant  Scores 

0.992 

0.598 

0.605 

0.962 

Mid  Machine 

Average  Values 

0.9996 

1.3030 

0.7997 

3.1164 

0.964 

Resultant  Scores 

1.000 

1.000 

0.749 

0.977 

Table  18:  CCS  KPP  Values  and  Scores 


C.  ARCHITECTURE 


It  was  demonstrated  that  the  architecture  constructs  shown  in  Figure  23  and  Figure  24, 
when  implemented  with  high  levels  of  automation  and  the  KPPs  shown  in  Table  18,  would 
provide  a  robust  CCS. 
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Figure  23:  High  Level  CCS  Architectural  Framework 
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LI  Architectural  Framework,  with  Generic  Sensor  Input 


Figure  24:  Detailed  CCS  Architectural  Construct 
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D.  COST  MODEL 


The  results  of  the  system  modeling  and  simulation  allow  a  reduction  in  manpower 
from  between  six  to  ten  operators  per  shift  in  the  current  VA  operational  base-line  to  four 
operators  per  shift,  with  an  average  utilization  of  34.1%.  This  52%  reduction  in 
manpower  utilization  will  potentially  save  the  Navy  $41.7  million  per  year. 

From  the  results  of  the  modeling  with  a  high  level  of  automation,  the  entire 
combat  system  personnel  per  platform  decreased  from  48  sailors  to  23.  The  four  shifts,  at 
four  people  per  shift  amounts  to  16  CCS  personnel.  Since  the  TM  rate  of  seven  is 
assumed  to  remain  constant  across  current  and  future  systems,  this  brings  the  total 
combat  system  billet  count  to  23  as  shown  in  Figure  25.  This  RTOC  amounts  to  a  total 
CCS  personnel  savings  of  25  per  platform  and  over  the  course  of  ten  years  amounts  to  a 
projected  yearly  savings  of  nearly  $42  Million  dollars  as  shown  in  Figure  26.  The  total 
projected  savings  over  the  course  of  ten  years  amounts  to  $465.9  million  as  shown  in 
Table  19. 


Figure  25:  Level  of  Automation  Personnel  Reduction. 
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VA  Fleet  Cost  Savings  Per  Enlisted  Sailor  Reduction 


FY 

Index 

1 

2 

3 

4 

5 

6 

7 

25 

2011 

1.0000 

$1,669,626 

$3,339,264 

$5,008,896 

$6,678,527 

$8,348,159 

$10,017,791 

$11,687,423 

$41,740,650 

2012 

1.0155 

$1,695,514 

$3,391,028 

$5,086,542 

$6,782,056 

$8,477,569 

$10,173,083 

$11,868,597 

$42,387,847 

2013 

1.0371 

$1,731,579 

$3,463,157 

$5,194,736 

$6,926,315 

$8,657,894 

$10,389,472 

$12,121,051 

$43,289,468 

2014 

1.0649 

$1,777,935 

$3,555,870 

$5,333,805 

$7,111,741 

$8,889,676 

$10,667,611 

$12,445,546 

$44,448,379 

2015 

1.0947 

$1,827,717 

$3,655,435 

$5,483,152 

$7,310,869 

$9,138,587 

$10,966,304 

$12,794,021 

$45,692,934 

2016 

1.1253 

$1,878,893 

$3,757,787 

$5,636,680 

$7,515,574 

$9,394,467 

$11,273,361 

$13,152,254 

$46,972,336 

2017 

1.1568 

$1,931,502 

$3,863,005 

$5,794,507 

$7,726,010 

$9,657,512 

$11,589,015 

$13,520,517 

$48,287,561 

2018 

1.1892 

$1,985,585 

$3,971,169 

$5,956,754 

$7,942,338 

$9,927,923 

$11,913,507 

$13,899,092 

$49,639,613 

2019 

1.2225 

$2,041,181 

$4,082,362 

$6,123,543 

$8,164,724 

$10,205,904 

$12,247,085 

$14,288,266 

$51,029,522 

2020 

1.2568 

$2,098,334 

$4,196,668 

$6,295,002 

$8,393,336 

$10,491,670 

$12,590,004 

$14,688,338 

$52,458,349 

Totals 

$18,637,866 

$37,275,745 

$55,913,617 

$74,551,489 

$93,189,361 

$111,827,234 

$130,465,106 

$465,946,659 

Table  19  VA  Fleet  Cost  Savings  Per  Enlisted  Sailor  Reduction 


E.  SUMMARY  OF  RESULTS 


Figure  26  depicts  the  graphical  summary  from  the  previous  cost  analysis.  The 
derived  savings  above  were  based  on  a  simple  average  DUC  (salary  and  other 
compensation)  across  all  of  the  enlisted  billets  in  Table  7.  An  average  of  DUC  was 
selected  to  simplify  the  analysis. 
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Figure  26  VA  Fleet  Cost  Savings  25  Sailor  Reduction 

It  can  be  seen  from  Figure  26  that  by  reducing  the  VA  submarine  force  between  4 
to  25  sailors  between  $6.7M  and  $41. 7M.  Extending  this  savings  over  a  ten  year  period 

can  result  in  a  potential  savings  between  $8.4M  to  $52. 5M. 
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IX  CONCLUSIONS 


The  intent  of  this  project  was  to  develop  a  construct  to  demonstrate  that  a  new 
approach  to  the  CCS  in  an  end-to-end  engagement  chain  context  and  high  level 
automation  can  reduce  the  CCS  TOC  to  the  Submarine  Force.  It  was  shown  that  effective 
SA  with  reduced  manpower  is  feasible  by  adopting  a  new  CONEMPS  that  achieves  the 
following:  (a)  allocating  the  various  engagement  chain  functions  to  a  combination  of 
people  and  machine,  (b)  changing  from  man-as-operator  to  machine-as-operator,  (c)  cross 
training  the  CC  crew  to  enable  the  use  of  personnel  pools  at  the  sensors  and  trackers. 

A  modified  SIMILAR  systems  engineering  model  was  used  to  create  the 
construct  and  define  the  system  functions  to  construct  an  ExtendSim®  model  to  evaluate 
the  system  under  stress  case  conditions.  Based  on  the  DOE,  the  ExtendSim®  model 
settings  were  adjusted  to  measure  the  KPPs  and  prove  that  the  CCS  can  be  operated  with 
a  crew  of  four  persons  per  shift.  To  achieve  this  level  of  personnel  in  the  future  system 
development,  a  new  approach  to  the  fundamental  operator  KSAs,  HSI  and  training 
systems  will  need  to  be  implemented  by  the  Submarine  Force. 

By  adopting  this  change  in  the  system  development  the  reduction  in  personnel 
from  (48)  to  (23)  can  save  the  Submarine  Force  on  the  order  of  a  $42  Million  dollars  in 
DUC  annually.  This  reduction  amounts  to  a  52%  reduction  in  manpower  utilization 
which  is  a  projected  savings  over  the  course  of  ten  years  amounts  to  $465.9  million 
dollars.  These  results  are  summarized  in  Table  20.  This  savings  does  not  account  for 
any  of  the  other  annual  unit  cost  savings  such  as  the  indirect  costs  (IC)  and  maintenance 
costs  (DIC  and  DDC).  The  operations  and  maintenance  requirement  assessment  of  the 
system  will  be  necessary  to  ensure  that  the  maintenance  of  the  equipment  does  not  burden 
the  personnel  based  on  the  overall  reduction  and  collateral  duties  such  as  damage  control. 
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Personnel  Per  Platform 

DUC 

VA  Fleet  Ten  Year 

Savings 

VA  Combat  System 

CCS 

Reduction 

$M 

48 

23 

52% 

42 

465.9 

Table  20  Summary  of  Reduction  Results 


The  results  of  the  ExtendSim®  modeling  upheld  the  reduction  in  personnel  in  the 
CCS  based  on  the  proposed  CONEMP.  The  modeling  showed  that  accounting  for 
automation  and  human  effectiveness  levels  would  produce  an  effective  CCS.  The  model 
settings  validated  that  the  engagement  chain  functions  can  be  accomplished  with  high 
level  automation  and  a  minimum  of  four  persons  per  shift. 

The  human  CCS  operator  has  to  act  in  a  more  supervisory  role,  taking  a  higher 
level  viewpoint  of  the  situational  picture.  This  allows  him  to  not  only  understand  a  small 
niche  of  data,  but  also  a  higher  level  of  information.  This  required  level  of  information 
certainty  demands  a  greater  level  of  trust  in  the  system,  which  has  yet  to  be  proven.  It  is 
imperative  that  the  human  not  be  removed  from  the  processing  loop  for  a  few  reasons 
including: 

1 .  The  socio-political  risk  is  too  great  for  a  fully  automated  armed  system. 

2.  Without  enough  interaction,  the  human  may  become  complacent,  relying 
on  automated  outputs  and  lowering  his  overall  situational  mental  model  correctness. 

Increasing  automation,  yet  keeping  the  man  in  the  processing  loop  with  sufficient 
workload  to  maintain  vigilance,  will  effectively  elevate  him  to  a  supervisory  role.  This 
allows  the  machines  to  act  in  an  advisory  role  to  the  human,  affording  trust  in  the  data. 
To  support  that  supervisory  role,  the  human  needs  to  work  across  multiple  sensors.  The 
best  way  to  realize  these  conflicting  requirements  is  to  re-align  the  human  from  being  an 
expert  in  a  particular  job  to  becoming  a  more  highly  trained  “super”  operator  across 
several  tasks.  He  also  needs  to  work  in  a  “pool”  of  human  resources  that  can  be  applied 
as  required  by  the  situation. 
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Based  on  the  modeling  and  analysis  completed  during  this  project,  it  is  feasible  to 
implement  a  CCS  that  reduces  operational  cost  by  a  total  of  $465.9  million  dollars  over 
ten  years.  These  results  were  achieved  using  the  operational  values  listed  Table  21. 


%  Machine 

Number  of 
Operators 

Machine 

Effectiveness 

BBG 

Process 

NBG 

Process 

Visual 

Process 

ES 

Process 

Update 

Tracker 

Process 

TMA 

Process 

TMA 

Sensor 

Pool 

75 

75 

75 

25 

75 

75 

1 

3 

2 

Table  21  Operational  Results 


It  is  noted  that  the  machine  effectiveness  listed  in  Table  21  may  be  difficult  to 
achieve  due  to  the  investment  needed  to  develop  intelligent  systems  capable  of  expert 
operator  performance.  Thus,  further  analysis  was  performed  to  estimate  the  system 
performance  with  a  machine  effectiveness  assumed  to  be  one-half  and  equal  to  man 
effectiveness.  In  both  cases  it  was  shown  that  although  there  is  a  decrease  in  system 
performance,  the  change  is  small.  Therefore,  it  is  concluded  that  it  is  unnecessary  to  wait 
for  the  machine  effectiveness  to  achieve  200  percent  of  man  effectiveness  to  implement 
the  construct  presented. 
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RECOMMENDATIONS 


A.  ROADMAP  TO  IMPLEMENTATION 

In  order  to  implement  the  derived  construct,  there  are  required  refinements  to  the 
model,  as  well  as  certain  enabling  technologies  needed  to  implement  a  truly  integrated 
combat  system  that  supports  the  balance  of  man  to  machine  demonstrated  by  this  project. 
It  is  also  anticipated  that  significant  changes  in  acquisition  and  employment  of  the 
submarine  systems  will  be  required  in  order  realize  this  construct.  These  changes  can  be 
categorized  as  enabling  technologies,  follow-on  development  work,  and  organizational 
changes.  Changes  in  the  approach  to  acquisition  are  anticipated  but  are  beyond  the  scope 
of  this  technical  project. 

B.  ENABLING  TECHNOLOGIES 

The  detailed  hardware  design,  algorithm  development  and  system  architecture  are 
beyond  the  scope  of  this  project.  However,  factors  that  must  be  considered  in  the  design 
and  implementation  of  a  construct  that  would  support  the  future  SCSEP  construct  are 
addressed  in  the  following  sections. 

1.  Architectural  Elements 

The  inclusion  of  modem  components  and  forms  of  implementation,  such  as 
massively  parallel  processing,  self-learning,  fuzzy  logic,  genetic  and  heuristic  software 
elements  should  be  investigated  to  increase  system  effectiveness.  These  can  be 
introduced  as  adjunct  elements  that  override  the  current  approach  of  pre-programmed 
algorithms  that  exist  in  a  fixed  stove-piped  procedural  architecture. 

2.  Hardware 

Key  factors  in  the  selection  of  hardware  are  the  efficient  sharing  of  resources, 
commonality  and  redundancy.  Servers  that  can  be  utilized  for  diverse  processing 
functions  will  enable  efficient  use  of  the  hardware.  For  example,  processing  power 
allocated  to  imaging  and  ES  during  PD  operations  will  be  reallocated  to  sonar  processing 
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while  submerged.  Similarly,  while  preparing  for  and  at  PD,  these  same  processors  will 
reallocate  near  real  time  to  support  the  varying  sensor  loads. 

Commonality  in  the  hardware  will  be  crucial  both  to  support  the  allocation  of 
varying  sensor  loads  and  to  enable  the  concept  of  pooled  maintenance  divisions.  By 
utilizing  common  hardware  across  the  platform,  the  need  for  specialized  technicians  will 
be  reduced. 

Redundancy  in  the  hardware  and  the  failure  modes  of  the  architecture  must  be 
fully  considered  and  addressed  in  the  design  of  the  future  SCSEP.  Due  to  the  severe 
consequences  involved,  all  hardware  must  be  fail  safe  and  critical  hardware  must  be 
designed  to  allow  for  manual  operations  in  the  event  of  failure.  The  failure  modes  must 
be  such  that  failures  allow  the  operator  to  maintain  SA  by  a  gradual  transition  from 
higher  to  lower  level  of  automation. 

3.  Algorithm  Development 

The  algorithms  developed  to  support  the  SCSEP  construct  must  be  functionally 
verified  and  reliable.  As  previously  discussed,  the  crew  must  trust  the  automation.  Since 
the  undersea  acoustic  performance  is  environment  dependent,  both  in-lab  and  at  sea 
testing  would  be  required  to  certify  the  algorithms.  Similar  to  the  hardware,  the  algorithm 
development  must  produce  software  with  high  reliability  and  graceful  failures. 

Redundancy  in  the  functions  will  provide  reliability  both  in  the  execution  of  the 
functions  and  in  the  error  checking  of  the  functions.  Multiple  independent  algorithms 
error  checking  each  other  will  provide  a  more  robust  design  that  will  serve  to  reduce 
single  points  of  failure  and  will  better  account  for  the  extremes  of  the  operating 
environments. 

Limitations  of  the  algorithms  must  be  accounted  for  in  the  failure  modes.  The 
algorithms  must  account  for  their  inherent  limitations  and  provide  a  graceful  transition  to 
lower  levels  of  automation  as  system  limitations  are  approached  and  exceeded.  Fail  safe 
and  operational  requirements  must  be  considered  and  appropriate  requirements  allocated 
to  the  algorithms  during  the  design  stage. 
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4.  Machine  Effectiveness 

Machine  effectiveness  is  a  combination  of  algorithm  and  machine  efficiency,  and 
algorithm  accuracy.  Table  16  shows  that  there  are  small  changes  in  system  performance 
relative  to  machine  effectiveness  when  the  machine  is  set  to  a  range  of  0.5  to  2  times 
human  effectiveness.  Therefore,  this  construct  can  be  introduced  once  the  lower  level  of 
machine  effectiveness  is  achieved. 

The  perception  engine  must  be  better  able  to  perform  computations  and  output 
advisory  contact  information.  While  automation  is  partially  implemented  now  in  the  form 
of  trackers  and  bell  ringers,  the  scope  of  automation  needs  to  be  greatly  expanded  in 
order  to  enable  the  suggested  architecture.  This  research  is  ongoing  at  a  sub-system  and 
stove-piped  level,  but  advances  in  this  area,  across  sensors  will  be  required. 

In  order  to  allow  the  human  to  trust  the  machine,  algorithms  must  be  advanced  to 
lower  the  number  of  both  type  I  (false  alarms)  and  type  II  errors  (missing  a  real  contact). 
As  suggested  in  the  Model  Design  section,  “the  expectation  of  detecting  a  genuine  target 
must  be  at  least  ten  times  the  expectation  of  encountering  a  false  alarm”  [Chapel,  2010], 
Without  this  low  false  alarm  technology,  the  automation  will  not  be  trusted,  resulting  in 
the  human  trying  to  do  too  much  of  the  work  himself.  In  this  case,  the  operators  will 
either: 


•  Shed  tasks,  thus  becoming  less  effective,  or 

•  Miss  needed  contacts,  or 

•  Contacts  will  be  prosecuted  cursorily. 

This  could  result  prioritizing  a  contact  incorrectly. 

5.  Human  System  Integration 

HSI  is  a  critical  component  of  any  design.  However,  in  a  highly  automated 
design  such  as  the  SCSEP,  the  effective  integration  of  the  human  operators  and  the 
automation  is  particularly  essential.  The  data  processed  by  the  automation  must  be 
displayed  as  actionable  information  to  the  operators.  In  addition  to  the  data  processing 

algorithms,  automation  to  prevent  and  detect  human  error  will  be  a  key  factor  in  a 
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successful  design  to  support  reduced  manning.  With  low  levels  of  automation,  the 
increased  manning  levels  provide  multiple  eyes  checking  the  information  processed  by 
the  operators  and  the  decisions  made  by  the  supervisors.  With  reduced  manning,  the 
automation  must  account  for  this  element  of  human  error  checking.  Trip  wires  should  be 
determined  and  bell  ringers  implemented  to  alert  the  supervisor  when  potential  human 
error  is  detected. 

C.  OTHER  FOLLOW-ON  WORK 

1.  More  Detailed  Functional  Decomposition 

For  the  model,  only  the  Level  1  functions,  as  described  in  the  functional 
decomposition  shown  in  Table  3and  Figure  7  were  considered.  This  work  could  be 
extended  to  a  more  detailed  look  at  the  tasks  and  thus  perhaps  improved  effectiveness 
requirements  for  implementation  of  automation. 

Figure  15  shows  the  distributions  used  for  TMA  and  Data  Fusion.  While  it  was 
outside  the  scope  of  this  report  to  actually  implement  these  algorithms,  the  distributions 
used  were  derived  from  interviews  and  personal  experience.  Further  research  could  be 
performed  to  confirm  or  update  these  numbers,  including  research  that  would  investigate 
new  techniques  and  algorithms  that  might  improve  these  values.  This  would  involve 
looking  at  newer  techniques  for  tracking  contacts,  or  fusing  data  across  sensors. 

2.  Expand  the  “Projection”  Portion  of  SA  into  the  Effectiveness  Equation 

In  the  decomposition  of  SA  “projection”,  it  was  determined  that  to  actually 
simulate  the  “act”  portion  of  the  engagement  chain  would  be  very  complicated.  This  is 
due  to  the  fact  that  models  of  people  would  be  required  to  simulate  performing  actions 
and  interpret  mission  requirements.  For  example,  when  the  platform  comes  across  a 
contact,  it  could  shoot  or  maneuver.  Whether  to  shoot  or  not  requires  understanding  of 
the  mission:  Is  the  object  of  the  mission  to  actually  eliminate  the  contact?  If  not,  there 
are  ramifications  to  doing  so.  Indeed,  it  is  an  assumption  that  the  submarine  command 
will  make  the  correct  decision  given  good  information.  A  method  of  advancing  the 
mission  requirements  into  the  model,  or  even  a  better  value  for  how  well  command 
responds  to  a  situation,  could  be  investigated. 
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3.  Additional  Sensors 

Only  certain  sensors  were  considered,  partially  to  keep  this  report  unclassified. 
Additional  sensors  could  be  considered  such  as  active  sonar.  As  another  example,  reliable 
ship-to-ship  communications  might  be  considered  to  share  sensor  and  contact  data  across 
platforms.  Once  these  are  considered,  the  results  from  the  sensors  could  be  improved  by 
allowing  directed  searches,  or  being  cued  to  specific  contacts.  This  is  an  extension  of  the 
“fusion  pipeline”  discussed  above,  and  the  net  effect  would  be  a  consolidated  federation 
of  networked  sensors. 

4.  More  detailed  Understanding  of  the  Cost  of  Manning 

Consider  all  the  costs  involved  in  manning,  not  just  the  direct  costs.  There  are 
many  costs  to  manpower:  training,  room,  board,  equipment,  salary,  support  (secretaries, 
human  resources  offices),  overhead  (heat,  light  and  paper),  etc.  As  a  result  of  the  reduced 
personnel,  overall  reductions  in  cost  can  be  realized  in  terms  of  onboard  support 
equipment  such  as  computer  infrastructure  and  tooling;  as  well  as  a  reduction  in  land 
based  administrative  support  requirements. 

5.  Other  Possible  Related  RTOC  Sources 

In  addition  to  the  manpower  cost  savings,  there  are  other  areas  where  cost  savings 
can  be  gained  as  part  of  technology  upgrades  to  the  CCS.  As  the  manpower  reduction  is 
realized,  there  could  be  a  corresponding  decrease  in  the  needed  foot  print: 

•  Reduced  habitability;  rack,  oxygen  generation,  personal  safety  equipment, 
provision  storage  and  galley  space,  or 

•  Reduced  number  of  operator  workstations. 

This  reduction  in  habitability  and  equipment  footprint  would  provide  much  needed  areas 
for  additional  hardware,  cabling  and  systems  needed  to  provide  a  reliable  automated 
CCS. 
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D.  IMPLEMENT  FINAL  FUNCTIONAL  ALLOCATION,  ACCORDING  TO 


ORGANIZATIONAL  CHANGES 

It  is  anticipated  that  the  reduced  manning  and  changes  in  employment  will  require 
changes  in  the  crew  organization.  Currently  the  crew  is  organized  by  departments,  which 
are  further  organized  into  divisions.  Departments  are  led  by  officers  who  serve  as 
department  heads.  The  department  heads  are  responsible  for  the  divisions  that  make  up 
their  department.  The  divisions  are  managed  by  junior  officers  who  serve  as  division 
heads.  A  senior  enlisted  crew  member  or  lead  petty  officer  (LPO)  coordinates  with  the 
division  head  to  manage  the  enlisted  crew  that  comprise  the  division.  The  division  heads 
manage  both  the  operational  and  maintenance  duties  of  the  crew.  The  current 
organization  is  based  partly  on  the  stove  piped  employment  of  the  sensors.  The  left  of 
Figure  27  shows  the  current  divisions  on  the  VA  class  submarine  [USS  New  Mexico, 
2011],  It  is  anticipated  that  the  organization  of  the  divisions  will  be  modified  resulting  in 
fewer  divisions.  Additionally,  it  is  anticipated  that  the  division  organization  will  be 
further  subdivided  into  operational  divisions  and  maintenance  divisions  as  shown  on  the 
right  of  Figure  27. 


Current  Divisions 
(Operational  and  Maintenance) 


Communications 

LAN 

Sonar 

Fire  Control 
Navigation 


Potential  Operational  Division 


Combat  System  Operator 


Potential  Maintenance  Divisions 


Sensors 

Processors/LAN 


Figure  27  Personnel  Breakdown 
1.  Operational  Divisions  and  Maintenance  Divisions 

The  enlisted  crew  members  responsible  for  the  combat  systems  have  two  primary 
functions,  which  are  to  serve  as  operators  and  maintainers.  Currently  the  enlisted  fleet 
that  comprises  the  Submarine  Electronics  /  Computer  Field  (SECF)  is  organized  by 
specialty  ratings  identified  as  Sonar  Technician  Submarine  (STS),  Fire  Control 
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Technician  (FT),  and  Electronics  Technician  (ET).  The  ET  rate  is  further  divided  into 
Navigation  (ET-Nav)  and  communication  (ET-Comms).  [Powers,  2011]  The  crew  that 
fill  these  ratings  have  specialized  training  based  on  the  unique  sensors  they  maintain  and 
operate.  Just  as  the  combat  system  will  be  re-architected  to  eliminate  the  stove  piped 
sensors  employment,  the  divisions  will  need  to  be  organized  to  eliminate  the  stove  piped 
operator  concept.  A  pool  of  independent  sensor  operators  must  be  available.  These 
operators  will  adjust  their  focus  as  the  operational  environment  dictates.  For  example, 
while  submerged  deep  in  an  environment  saturated  by  acoustic  contacts,  operators  will 
process  the  automation  representation  of  the  sonar  sensor  information.  As  the  submarine 
arrives  at  PD,  and  visual  and  ES  contacts  increase,  more  operator  focus  could  potentially 
utilize  the  automation  output  of  the  fused  sensor  data  from  all  available  sensors.  The  old 
CONEMP  dependent  on  Sonar  operators  and  ES  operators  and  Imaging  operators  will  be 
replaced  by  a  revised  CONEMP,  which  involves  versatile  combat  system  operators. 

Similarly,  the  officers  and  supervisors  will  need  to  adapt  to  supervise  the  combat 
system  as  opposed  to  supervising  sensor  specific  data.  The  legacy  Fire  Control 
Technician  of  the  Watch  (FTOW)  and  Sonar  Supervisor  roles  would  be  replaced  by  a 
single  combat  system  supervisor  in  low  contact  density  environments.  As  the  contacts  in 
the  environment  increase,  an  additional  combat  system  supervisor  could  be  called  upon 
from  the  supervisor  pool. 

The  focus  of  this  report  is  on  the  operational  aspects  of  the  combat  system  but  the 
required  maintenance  cannot  be  totally  ignored.  Therefore,  it  is  anticipated  that  the 
maintenance  divisions  will  be  reorganized  into  specialized  sensor  specific  maintainers 
and  sensor  diagnostic  computer  system  and  LAN  maintainers.  This  organization  allows 
the  reduction  in  manning  realized  by  replacing  the  stove  piped  specialized  operators  not 
to  impact  the  maintainability  of  the  equipment.  The  same  pooled  operators  will  support 
maintenance  of  the  common  hardware  utilized  for  backend  sensor  processing.  Due  to  the 
uniqueness  of  the  sensors,  it  is  anticipated  that  a  small  specialized  sensor  maintenance 
division  will  still  be  required  but  reduction  in  the  overall  maintenance  force  will  still  be 
realized. 
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2.  Changes  in  Training 

As  discussed,  the  roadmap  must  include  migration  from  stove-piped  division  of 
operators  within  the  SECF  to  a  pooled  SECF  division  of  operators.  The  enlisted 
technicians  would  need  to  be  cross-trained  as  operators.  This  pool  of  operators  must  also 
be  trained  to  function  as  a  pool  of  maintainers  for  the  common  equipment.  While  the 
need  for  specialists  to  support  unique  sensor  maintenance  cannot  be  eliminated,  the 
roadmap  must  reduce  the  number  of  specialized  roles. 

3.  Changes  in  Culture 

Automation  is  implemented  for  two  primary  reasons:  to  reduce  operator  workload 
and  to  increase  vessel  safety  and  effectiveness.  To  be  successful,  the  end  user  must 
effectively  work  with  the  automation,  using  and  trusting  the  tools,  but  also  understanding 
what  automation  is  doing.  If  the  user  does  not  trust  the  automation  then  it  will  not  be 
accepted  and  will  not  be  used.  Conversely  if  the  user  places  too  much  trust  in  the 
automation  then  catastrophe  can  result  if  the  automation  fails  or  when  the  limits  of  the 
automation  are  exceeded  and  manual  control  must  be  regained  instantly. 

“The  Navy  is  a  service  of  custom  and  tradition”  [Bundy,  2010].  The  roadmap  to 
implementation  must  address  the  changes  in  custom  and  tradition,  which  will  be 
represented  by  the  elimination  of  stove-piped  sensor  based  operators  and  the  introduction 
of  pooled  combat  system  operators.  It  is  anticipated  that  time  will  be  required  for  the 
crews  to  become  accustomed  to  the  changes  represented  by  this  new  CONEMP.  This  is 
similar  to  the  time  that  was  required  for  the  crews  to  become  familiar  with  and  accepting 
of  visual  sonar  waterfalls  as  opposed  to  aural  sonar  and  the  transition  from  optical  based 
imaging  to  sensor  based  imaging.  The  critical  factor  required  for  acceptance  of  the 
change  will  be  trust  in  the  new  CONEMP  and  trust  in  the  algorithms  that  make  the  new 
CONEMP  realizable. 

Paramount  to  building  trust  in  the  automation  CONEMP  is  data  confidence.  For 
man  to  trust  the  automation  he  needs  to  understand  why  the  algorithm  is  making  any 
given  recommendations.  Man  usually  trusts  the  algorithm  if  it  is  recommending  low  risk 
maneuvers.  It  is  when  the  algorithm  recommends  something  that  is  risky  that  man 
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questions  it.  Man  then  spends  time  questioning  the  algorithm  instead  of  reacting.  The 
man  needs  to  be  able  to  sense  when  to  question  what  the  automation  is  reporting. 
Otherwise,  if  it  breaks  he  will  not  know  how  to  react  and  he  will  not  know  when  it  has 
failed.  An  example  is  autopilot  on  airplanes  -  the  pilots  trust  the  automation  but  they  also 
are  trained  to  know  when  they  need  to  take  over. 

In  order  to  support  a  gradual  shift  in  culture,  a  phased  approach  could  be 
implemented  where  the  levels  of  automation  increase  as  the  data  confidence  and  the 
Fleet’s  trust  of  automation  increases.  The  more  the  automation  increases,  the  more 
critical  it  will  become  for  fail  safe  implementations,  which  provide  redundancy  not  just  in 
the  hardware  but  also  in  the  software  to  eliminate  single  point  failures.  Studies  have 
shown  that  when  automation  failures  occur  that  require  the  user  to  return  to  full  manual 
control,  the  higher  the  Level  of  Automation  (LOA)  at  the  time  of  the  failure,  the  more 
difficult  it  is  for  the  operator  to  return  to  manual  control  [Di  Nocera,  2005],  With  a  highly 
automated  CCS,  it  could  be  catastrophic  if  the  automation  failed  and  the  crew  had  limited 
time  to  regain  control  and  regain  SA  at  the  level  required  for  full  manual  control. 
Therefore,  the  automation  that  will  allow  the  reduction  in  personnel  must  be  implemented 
to  allow  graceful  degradation  and  must  be  fail-safe. 
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XIII  APPENDIX  A:  ACRONYMS 


AIS 

Automatic  Identification  System 

AoA 

Analysis  of  Alternatives 

ARIS 

Automated  Readiness  Information  System 

BBG 

Broadband  Gram 

C2 

Command  and  Control 

CBO 

Congressional  Budget  Office 

CC 

Combat  Control 

CCE 

Combat  Control  Efficiency 

CCF 

Completeness  of  Comprehension  (fusion) 

CCS 

Combat  Control  System 

CCT 

Correctness  of  Comprehension  (TMA) 

CHENG 

Chief  Engineer 

CM 

Contacts  Maintained 

COC 

Contact  of  Concern 

COI 

Contact  of  Interest 

CONEMP 

Concept  of  Employment 

CONOPS 

Concept  of  Operations 

CRS 

Congressional  Research  Services 

CS 

Combat  System 

DDC 

Direct  Depot  Cost 

DEVRON-12 

Submarine  Development  Squadron  Twelve 

DIC 

Direct  Intermediate  Cost 

DoDAF 

Department  of  Defense  Architecture  Framework 

DOE 

Design  of  Experiment 

DTIC 

Defense  Technical  Information  Center 

DUC 

Direct  Unit  Cost 

EFFBD 

Enhance  Functional  flow  Block  Diagram 

ES 

Electronic  Sensor 

ESM 

Electronic  Surveillance  Measure 

ET 

Electronics  Technician 

ET-Comms 

Communications 
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ET-Nav 

Navigation 

FBCM 

Fused  Bearing  Confidence  Mean 

FRCM 

Fused  Range  Confidence  Mean 

FSM 

Fusion  Score  Mean 

FT 

Fire  Control  Technician 

FTOW 

Fire  Control  Technician  of  the  Watch 

GAO 

Government  Accountability  Office 

HSI 

Human  System  Interface 

IC 

Indirect  Cost 

IDEFO 

Integration  Definition  for  Process  Modeling 

IFF 

Identification  Friend  or  Foe 

INCOSE 

International  Council  of  Systems  Engineers 

IPR 

In  Process  Review 

ISR 

Intelligence,  Surveillance,  Reconnaissance 

KPP 

Key  Performance  Parameter 

KSA 

Knowledge,  Skills  and  Abilities 

FAN 

Focal  Area  Network 

FOA 

Fevel  of  Automation 

EOS 

Fine  of  Sight 

FPO 

Feading  Petty  officer 

ME 

Mission  Effectiveness 

MFO 

Mind-like  Objects 

MOE 

Measures  of  Effectiveness 

MOM 

Measures  of  Merit 

MOP 

Measures  of  Performance 

NAVSEA 

Naval  Sea  Systems  Command 

NBG 

Narrowband  Gram 

NPS 

Naval  Postgraduate  School 

NSMRL 

Naval  Submarine  Medical  Research  Faboratory 

NSWCCD 

Naval  Surface  Warfare  Center  Carderock  Division 

NUW  CDIVNPT 

Naval  Undersea  Warfare  Center  Division  Newport 

NWC 

Naval  War  College 

OOD 

Officer  of  the  Deck 
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OV-1 

Operational  View  1 

PARM 

Program  Acquisition  Resource  Manager 

PBB 

Passive  Broadband 

PD 

Periscope  Depth 

PEO  SUB 

Program  Executive  Office,  Submarines 

PNB 

Passive  Narrowband 

RCN 

Rating  Control  Number 

ROE 

Rules  of  Engagement 

RTOC 

Reduced  Total  Ownership  Cost 

SA 

Situational  Awareness 

SCSEP 

Submarine  Combat  Systems  Engineering  Project 

SEAL 

Sea,  Air  and  Land 

SECF 

Submarine  Electronics  /  Computer  Field 

SIMILAR 

State,  Investigate,  Model,  Integrate,  Launch,  Assess  Re-evaluate 

SME 

Subject  Matter  Expert 

SNR 

Signal  to  Noise  Ratio 

SOF 

Special  Operations  Forces 

STS 

Sonar  Technician  Submarine 

TM 

Torpedo  Man 

TMA 

Target  Motion  Analysis 

TOC 

Total  Ownership  Cost 

TPM 

Technical  Performance  Measure 

UNTL 

Universal  Naval  Task  List 

USN 

United  States  Navy 

VA 

Virginia 

VAMOSC 

Visibility  and  Management  of  Operating  and  Support  Costs 

YHF 

Very  High  Frequency 
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XIV  APPENDIX  B:  ASSUMPTIONS 


The  following  section  outlines  the  various  assumptions  that  were  made 
throughout  the  engineering  effort. 

A.  SCSEP  PARADIGM  SHIFT 

An  assumption  of  this  SCSEP  project  is  that  a  paradigm  shift  will  be  required  in 
the  way  the  systems  are  developed,  utilized  and  organized,  resulting  in  higher  “people- 
ware”  efficiencies.  The  current  stovepiped  programs  of  record  independently  create 
acquisition  systems  as  part  of  new  construction  and  modernization  programs  with 
differing  development  schedules.  In  addition  to  the  systems  being  developed 
independently,  the  operator  training  products  are  developed  independent  of  other 
systems. 

To  achieve  the  goal  of  a  single  end-to-end  system,  the  consolidation  of  the  system 
development  would  need  to  occur  to  focus  specifically  on  creating  a  single  CCS  system. 
Interfaces  for  a  consolidated  CCS  would  focus  on  the  data  flow  to  ensure  that  the  correct 
information  is  exchanged  between  system  functions.  The  sensor  operator  interface  would 
need  to  be  created  to  present  situational  knowledge  instead  of  low  level  scientific  data. 

To  operate  the  system,  CCS  sensor  operators  will  need  to  be  trained  in  technical 
areas  that  are  currently  billeted  across  several  current  shipboard  systems.  The  operator 
pool  concept  that  is  introduced  in  this  project  leverages  the  cross  trained  operator  KSAs 
to  achieve  a  better  utilization  of  the  sensor  operator’s  time  “people-ware”  efficiencies. 

B.  HUMAN  VERSUS  MACHINE  TRADESPACE 

The  intent  of  this  project  is  to  assess  the  feasibility  of  replacing  the  functionality 
of  the  CCS  with  machine  based  processing  by  varying  the  functional  human  versus 
machine  ratio  in  the  detection,  identification,  tracking,  decision,  engagement  and 
assessment  of  the  submarine  engagement  chain.  This  effort  results  in  an  overall  analysis 
into  the  staffing  levels  for  the  Combat  System  including  STS,  FT,  TM,  ET  and 
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Radioman.  With  a  shipboard  compliment  of  (48)  CS  persons,  a  reduction  in  personnel 
could  achieve  a  significant  cost  savings. 

C.  FOCUS  ON  COMBAT  CONTROL  ONLY 

To  quantify  the  RTOC  the  analyses  that  are  contained  within  this  report  are 
focused  on  the  level  of  manning  specific  to  the  combat  system.  The  team  realizes  that 
there  are  other  responsibilities  associated  with  non-combat  system  related  functions  such 
as  damage  control,  equipment  maintenance,  messing,  berthing  and  general  housekeeping. 
The  impacts  to  the  overall  reduction  outside  the  combat  system  are  not  considered.  In 
addition,  the  team  assumes  that  there  will  be  no  negative  consequences  incurred  as  a 
result  of  reducing  personnel. 

The  TM  rate  was  not  included  as  part  of  the  overall  reduction  in  the  CCS  operator 
pool  calculations.  The  focus  was  on  the  inboard  processing  of  the  sensor  data  up  to  the 
point  of  decisions  to  act,  therefore  all  components  of  the  weapons  programming, 
placement  and  launch  control  were  omitted  from  this  project.  Also,  the  operations 
perspectives  of  the  CCS  were  identified  with  respect  to  personnel  only.  The  maintenance 
activities  and  approach  will  need  to  be  evaluated  in  a  similar  method  to  substantiate  the 
final  reduction  in  Submarine  CCS  personnel. 

D.  VIRGINIA  CLASS  SUBMARINE  IS  OUR  COMPARISON  BASELINE 

The  RTOC  analysis  recognizes  that  the  physical  dimensions  of  the  submarine 
platform  are  centered  on  the  shipboard  complement  of  crewmembers.  The  assumption  for 
sake  of  simplicity  is  that  the  current  Virginia  Class  submarine  is  the  baseline  hull  design 
and  that  any  reduction  of  hull  size  as  a  result  of  manning  reduction  is  left  for  future 
analysis  and  is  beyond  the  scope  of  this  project. 

E.  CURRENT  SENSOR  SUITE  PERFORMANCE  NOT  CONSIDERED 

This  report  assumes  that  the  outboard  sensor  suite  provides  the  necessary  signal  to 
noise  ratio  visual  and  frequency  cues  needed  to  exceed  the  detection  thresholds  of  the 
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receiving  systems.  The  variability  in  the  contact  detections  are  representative  of  gaining 
and  losing  contacts  and  are  indicative  of  environmental  or  kinematic  effects  of  the  target. 

Improvements  to  detection  algorithms  and  sensor  display  of  sensor  data  are  not 
within  scope  of  this  project. 

F.  CURRENT  SENSOR  SUITE  PERFORMANCE  IS  ADEQUATE 

The  overall  system  development  and  implementation  of  the  sensors  will  provide 
the  necessary  performance  capacity  to  successfully  execute  the  combat  system 
requirements.  The  system  must  demonstrate  high  reliability  in  order  for  the  operators  to 
have  trust  in  the  system  output.  The  system  reliability  is  assumed  to  be  accounted  for  by 
expert  hardware  and  software  developers  with  sufficient  experience  levels,  data 
processing,  memory  capacity  and  algorithm  confidence.  These  attributes  are  necessary 
for  a  reliable  and  trustworthy  system. 

G.  PASSIVE  SONAR  ONLY 

This  project  team  considers  the  passive  sonar  system  and  omits  the  active  system 
to  simplify  the  overall  system  requirements  and  modeling  effort.  This  active  sonar  aspect 
of  the  combat  system  is  left  for  future  students  to  evaluate  and  assess  for  Fleet  based 
implementation. 

H.  WORST  CASE  CONTACT  DENSITY  IS  IN  THE  MEDITERRANEAN 

The  contact  arrival  rate  for  the  transit  from  the  homeport  to  the  combat  operating 
area  will  generate  the  highest  contact  loading  for  the  engagement  chain.  The  worst-case 
transit  point  identified  via  AIS  analysis  is  the  Straits  of  Gibraltar  where  the  shipping 
density  was  on  the  order  of  130  non- warship  contacts  per  hour.  Based  on  the  AIS 
contacts  shown  in  the  VT  Explorer®  software,  the  greatest  contact  density  appeared  to  be 
in  the  Mediterranean  Sea.  The  VT  Explorer®  software  was  used  to  determine  the  contact 
density  of  a  given  timeframe,  over  the  course  of  a  two  to  four  hour  period  in  the  Gibraltar 
area.  The  identified  vessels  were  binned  into  the  categories  of  pleasure,  merchant  and 
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warship.  The  resultant  data  was  used  as  an  input  into  the  SCSEP  model  as  a  stress  case 
for  the  contact  overload  condition. 

I.  EFFICIENCY  IS  A  BETTER  METRIC  THAN  SIMPLE  MANNING 
REQUIREMENTS 

Efficiency  is  the  overarching  metric  for  this  project.  It  is  a  function  of  contact 
processing  effectiveness  and  cost.  The  effectiveness  is  defined  as  the  ability  of  the 
combat  system  to  process  and  assess  the  threat  versus  non-threat  contacts  vice  the 
specifics  of  the  platform  performance  in  a  given  environment.  With  the  goal  of  RTOC, 
the  project  team  members  agreed  that  a  meaningful  metric  of  CC  efficiency  would  be 
defined  as  the  effectiveness  of  the  SCSEP  compared  to  the  number  of  people  operating 
the  equipment. 

J.  ONLY  DIRECT  COST  IN  MANNING  IS  CONSIDERED 

The  cost  model  accounts  for  the  direct  unit  cost  elements  that  are  related  to  each 
platform  and  does  not  account  for  the  potential  savings  in  terms  of  housing,  medical, 
sustainment  and  specialized  training  that  are  necessary  to  support  individuals  throughout 
their  careers.  The  approach  to  RTOC  estimation  for  cost  savings  was  to  address  reduction 
of  tangible  personnel  specific  costs  and  relate  that  cost  to  a  manning  reduction  in  CCS 
operations.  The  cost  savings  from  a  submarine  platform  Life  Cycle  and  Submarine  Force 
personnel  indirect  cost  perspective  was  considered  a  separate  topic  outside  the  scope  of 
this  systems  engineering  project.  The  indirect  cost  could  be  a  makeup  of  food,  clothing, 
health  care,  entertainment  and  other  necessities.  To  meet  the  timeline  and  to  fit  within 
the  scope  of  the  project  these  costs  were  left  to  future  studies. 

K.  COMMAND  PERSONNEL  WILL  MAKE  THE  CORRECT  DECISIONS 
GIVEN  THE  CORRECT  SITUATIONAL  PICTURE 

Part  of  the  engagement  chain  is  to  “engage”  or  “act”.  What  constitutes  that  action 
depends  upon  both  the  current  situation  and  the  mission  requirements.  Since  the  project 
team  cannot  know  a  priori  what  the  mission  requirements  are,  the  assumption  was  made 
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that  the  submarine  command  structure  has  enough  experience  to  make  the  right  action 
decision  if  provided  with  the  best  possible  current  situational  picture. 
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XV  APPENDIX  C :  SIMULATION  EXPERIMENTAL  DESIGN  AND 

ANALYSIS 


The  following  sections  address  the  experimental  analysis  that  was  conducted 
throughout  the  project. 

A.  SCSEP  FIRST  MODEL  RUN  OUTPUT  DISTRIBUTION  ANALYSIS 

The  input  parameter  set  for  the  DOE  were  in  accordance  with  the  process  in 
[Yang  2009],  The  DOE  functional  relationship  takes  the  form: 


(8) 


Converting  the  input  factors  for  the  SCSEP  project: 


(9) 


where  xi-xg  are: 

•  Man  BBG  Process, 

•  Man  NBG  Process, 

•  Man  Visual  Process, 

•  Man  ES  Process, 

•  Man  Update  Tracker  Process, 

•  Man  TMA  Process, 

•  Men  TMA, 

•  Men  Pool, 
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•  Machine  Effectiveness. 


The  above  inputs  represent  three  categories:  allocation  of  the  percentage  of 
personnel  assigned  to  a  particular  process,  the  number  of  personnel  available  and  the 
effectiveness  of  the  machine  system  when  compared  to  personnel. 

To  initiate  the  DOE,  the  inputs  to  the  Input,  Process,  Output  (IPO)  diagram  in 
Figure  28  was  created  to  illustrate  the  effects  in  the  model  and  the  output.  The  input 
ranges  for  the  model  are  shown  in  Table  22. 


Figure  28  SCSEP  IPO  Diagram 
1.  Custom  Factorial  Design 

The  two  level  full  factorial  designs  would  require  factors  on  the  order  of  29 
combinations,  or  a  total  of  512  combinations.  Yang  has  identified  that  higher  order 
interaction  effects  can  be  excluded  and  reduced  to  a  fractional  factorial  design  [Yang 
2009],  The  JMP9®  software  was  used  to  create  a  custom  DOE  model  representation  of 
the  SCSEP  project  using  the  Custom  DOE  worksheet.  To  create  the  worksheet  the 
software  requires  inputs  (factors)  and  outputs  (responses)  to  establish  the  model.  By 
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selecting  the  two  level  categorical  factors,  the  interactions  between  the  inputs  are 
calculated  as  shown  in  equation  (9)  above. 


Input  Factor 

Min 

Max 

Units 

Man  BBG  Process 

25 

75 

(%) 

Man  NBG  Process 

25 

75 

(%) 

Man  Visual  Process 

25 

75 

(%) 

Man  ES  Process 

25 

75 

(%) 

Man  Update  Tracker  Process 

25 

75 

(%) 

Man  TMA  Process 

25 

75 

(%) 

Men  TMA 

1 

4 

People 

Men  Pool 

3 

6 

People 

Machine  Effectiveness 

0.345 

2 

Table  22  SCSEP  DOE  Inputs 


JMP9®  was  used  to  design  a  custom  DOE  with  two  power  and  two  interaction 
levels.  The  minimum  and  maximum  input  levels  are  shown  in  Table  22.  The  fractional 
factorial  requires  the  lower  order  interactions  of  the  inputs  defined  in  Table  22.  The 
JMP9®  software  develops  the  model  parameters  by  determining  the  model  terms, 
resulting  in  forty-six  (46)  model  combinations  as  shown  in  Table  23.  Each  combination 
has  the  minimum  and  maximum  converted  into  (-1)  and  (+1)  notation  to  determine  the 
interactivity  of  the  factors. 


Combination 

F  actors/interactions 

1 

Man  BBG  Process 

2 

Man  NBG  Process 

3 

Man  Visual  Process 

4 

Man  ES  Process 

5 

Man  Update  Tracker  Process 

6 

Man  TMA  Process 

7 

Men  TMA 

8 

Men  Pool 

9 

Machine  Effectiveness 

l2 

(Man  BBG  Process)2 

22 

(Man  NBG  Process)2 

32 

2 

(Man  Visual  Process) 

42 

(Man  ES  Process)2 

52 

(Man  Update  Tracker  Process)2 
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Combination 

F  actors/interactions 

62 

(Man  TMA  Process)2 

72 

(Men  TMA)2 

82 

(Men  Pool)2 

92 

(Machine  Effectiveness) 

1 

2 

Man  BBG  Process 

Man  NBG  Process 

1 

3 

Man  BBG  Process 

Man  Visual  Process 

1 

4 

Man  BBG  Process 

Man  ES  Process 

1 

5 

Man  BBG  Process 

Man  Update  Tracker  Process 

1 

6 

Man  BBG  Process 

Man  TMA  Process 

1 

7 

Man  BBG  Process 

Men  TMA 

1 

8 

Man  BBG  Process 

Men  Pool 

1 

9 

Man  BBG  Process 

Machine  Effectiveness 

2 

3 

Man  NBG  Process 

Man  Visual  Process 

2 

4 

Man  NBG  Process 

Man  ES  Process 

2 

5 

Man  NBG  Process 

Man  Update  Tracker  Process 

2 

6 

Man  NBG  Process 

Man  TMA  Process 

2 

7 

Man  NBG  Process 

Men  TMA 

2 

8 

Man  NBG  Process 

Men  Pool 

2 

9 

Man  NBG  Process 

Machine  Effectiveness 

3 

4 

Man  Visual  Process 

Man  ES  Process 

3 

5 

Man  Visual  Process 

Man  Update  Tracker  Process 

3 

6 

Man  Visual  Process 

Man  TMA  Process 

3 

7 

Man  Visual  Process 

Men  TMA 

3 

8 

Man  Visual  Process 

Men  Pool 

3 

9 

Man  Visual  Process 

Machine  Effectiveness 

4 

5 

Man  ES  Process 

Man  Update  Tracker  Process 

4 

6 

Man  ES  Process 

Man  TMA  Process 

4 

7 

Man  ES  Process 

Men  TMA 

4 

8 

Man  ES  Process 

Men  Pool 

4 

9 

Man  ES  Process 

Machine  Effectiveness 

5 

6 

Man  Update  Tracker  Process 

Man  TMA  Process 

5 

7 

Man  Update  Tracker  Process 

Men  TMA 

5 

8 

Man  Update  Tracker  Process 

Men  Pool 

5 

9 

Man  Update  Tracker  Process 

Machine  Effectiveness 

6 

7 

Man  TMA  Process 

Men  TMA 

6 

8 

Man  TMA  Process 

Men  Pool 

6 

9 

Man  TMA  Process 

Machine  Effectiveness 

7 

8 

Men  TMA 

Men  Pool 

7 

9 

Men  TMA 

Machine  Effectiveness 

8 

9 

Men  Pool 

Machine  Effectiveness 

Table  23  DOE  Interaction  Factor  List 
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2.  KPPs  evaluated 


Percent 
Maintained 
Contacts  (KPPi) 

Fusion  Score  of 
Contributing 
Sensors  (KPP2) 

Accuracy 

(KPP3) 

#Men 

used 

(KPP4) 

Weight 

0.3191 

0.0383 

0.0960 

0.5470 

Threshold 

0.8 

0.8 

0.7 

8 

Evaluation  Score 

Goal 

0.999 

0.95 

0.9 

3 

Table  24  KPP  Assessment 


3.  Output  Responses  Used  To  Evaluate  the  Runs 

To  calculate  the  output  values,  the  following  four  responses  were  determined  by 
the  DOE  model: 


•  Percent  maintained  contacts  (CM) 

•  Completeness  of  Comprehension:  Fusion  (CCF) 

•  Correctness  of  Comprehension:  TMA  (CCT) 

•  Number  of  men  required  per  shift  (n) 


KPP 

ExtendSim®  Output 

Percent 

Maintained 

Contacts 

Fusion  Score 
of 

Contributing 

Sensors 

Accuracy 

Measure 

# 

Men 

used 

#  Men  TMA 

X 

#  Men  Pool 

X 

Fusion  Score  Mean  (CCF) 

X 

BBG  Status  1 

X 

BBG  Status  2 

X 

NBG  Status  1 

X 

NBG  Status  2 

X 

NBG  Status  3 

X 

Visual  Status  2 

X 

Visual  Status  3 

X 

Visual  Status  4 

X 

Bearing  Confidence  Mean 

X 
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KPP 

ExtendSim®  Output 

Percent 

Maintained 

Contacts 

Fusion  Score 
of 

Contributing 

Sensors 

Accuracy 

Measure 

# 

Men 

used 

Range  Confidence  Mean 

X 

BBG  Excessive  Wait 

(stale  target) 

X 

NBG  Excessive  Wait 

(stale  target) 

X 

Visual  Excessive  Wait 

(stale  target) 

X 

TMA  Utilization  mean(%) 

X 

Pool  Utilization  mean(%) 

X 

Table  25  Mapping  Model  Components  to  the  KPPs  Modeled 
The  DOE  resulted  in  the  Table  26. 


112 


Run 

%  Man 

%  Man 

%  Man 

%  Man  ES 

%  Man 
Update 
Tracker 

%  Man 

#  Men 

#  Men 

Machine 

Resulting 

BBG 

NBG 

Visual 

TMA 

TMA 

Pool 

Effectiveness 

Score 

1 

0.25 

0.5 

0.25 

0.25 

0.25 

0.75 

4 

6 

2 

0.68361845 

2 

0.25 

0.75 

0.5 

0.75 

0.25 

0.75 

4 

6 

0.345 

0.57197517 

3 

0.75 

0.75 

0.25 

0.25 

0.25 

0.25 

1 

3 

2 

0.90252514 

4 

0.5 

0.5 

0.5 

0.75 

0.75 

0.75 

3 

3 

2 

0.7422604 

5 

0.75 

0.5 

0.75 

0.75 

0.25 

0.75 

1 

6 

0.345 

0.61825469 

6 

0.5 

0.75 

0.75 

0.75 

0.25 

0.25 

3 

3 

0.345 

0.82667932 

7 

0.75 

0.25 

0.5 

0.25 

0.25 

0.75 

1 

6 

2 

0.69662336 

8 

0.75 

0.75 

0.5 

0.75 

0.25 

0.25 

1 

6 

1.1725 

0.66736508 

9 

0.25 

0.25 

0.75 

0.25 

0.75 

0.25 

3 

3 

2 

0.90247607 

10 

0.75 

0.25 

0.75 

0.75 

0.5 

0.5 

3 

5 

0.345 

0.59064209 

11 

0.75 

0.5 

0.75 

0.25 

0.25 

0.25 

3 

6 

2 

0.75329901 

12 

0.25 

0.75 

0.25 

0.75 

0.75 

0.25 

1 

3 

2 

0.84486423 

13 

0.25 

0.25 

0.25 

0.5 

0.25 

0.75 

1 

6 

0.345 

0.63708733 

14 

0.75 

0.75 

0.75 

0.25 

0.75 

0.75 

3 

6 

0.345 

0.46236265 

15 

0.75 

0.75 

0.75 

0.75 

0.75 

0.75 

1 

6 

2 

0.49346882 

16 

0.25 

0.5 

0.75 

0.75 

0.5 

0.25 

4 

3 

2 

0.85851432 

17 

0.75 

0.75 

0.25 

0.25 

0.75 

0.5 

4 

3 

2 

0.75819208 

18 

0.25 

0.25 

0.75 

0.75 

0.75 

0.75 

1 

3 

1.1725 

0.8076227 

19 

0.25 

0.25 

0.75 

0.75 

0.25 

0.75 

3 

6 

2 

0.7079838 

20 

0.75 

0.25 

0.25 

0.25 

0.25 

0.25 

1 

6 

0.345 

0.72924527 

21 

0.75 

0.25 

0.5 

0.5 

0.25 

0.5 

4 

3 

1.1725 

0.84760856 

22 

0.75 

0.25 

0.25 

0.5 

0.75 

0.25 

1 

6 

2 

0.59785916 

23 

0.5 

0.75 

0.25 

0.75 

0.75 

0.75 

1 

5 

0.345 

0.5629718 

24 

0.25 

0.75 

0.75 

0.25 

0.25 

0.75 

1 

3 

2 

0.80890232 

25 

0.25 

0.25 

0.25 

0.75 

0.25 

0.5 

4 

3 

0.345 

0.80740589 
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Run 

%  Man 

%  Man 

%  Man 

%  Man  ES 

%  Man 
Update 
Tracker 

%  Man 

#  Men 

#  Men 

Machine 

Resulting 

BBG 

NBG 

Visual 

TMA 

TMA 

Pool 

Effectiveness 

Score 

26 

0.75 

0.75 

0.75 

0.25 

0.75 

0.25 

1 

3 

1.1725 

0.80907624 

27 

0.25 

0.75 

0.75 

0.25 

0.25 

0.5 

1 

6 

0.345 

0.64193381 

28 

0.75 

0.25 

0.75 

0.75 

0.25 

0.25 

1 

3 

2 

0.90310743 

29 

0.75 

0.25 

0.75 

0.75 

0.75 

0.25 

4 

6 

2 

0.57892349 

30 

0.25 

0.25 

0.5 

0.75 

0.75 

0.25 

1 

6 

0.345 

0.53386447 

31 

0.75 

0.75 

0.25 

0.25 

0.25 

0.75 

4 

5 

0.345 

0.63999783 

32 

0.25 

0.75 

0.25 

0.25 

0.75 

0.75 

1 

6 

2 

0.52355981 

33 

0.75 

0.25 

0.25 

0.75 

0.75 

0.75 

4 

6 

0.345 

0.44768588 

34 

0.25 

0.75 

0.75 

0.25 

0.75 

0.5 

4 

3 

0.345 

0.71852303 

35 

0.25 

0.25 

0.75 

0.25 

0.25 

0.25 

1 

3 

0.345 

0.87152614 

36 

0.75 

0.75 

0.75 

0.25 

0.25 

0.25 

4 

3 

0.345 

0.84536655 

37 

0.5 

0.75 

0.75 

0.25 

0.5 

0.75 

4 

6 

2 

0.48522334 

38 

0.5 

0.25 

0.25 

0.75 

0.25 

0.25 

4 

6 

2 

0.76749173 

39 

0.75 

0.25 

0.25 

0.75 

0.75 

0.25 

4 

3 

0.345 

0.84023196 

40 

0.25 

0.75 

0.75 

0.75 

0.5 

0.75 

1 

3 

0.345 

0.70342472 

41 

0.25 

0.25 

0.75 

0.25 

0.25 

0.75 

4 

3 

0.345 

0.76827823 

42 

0.75 

0.25 

0.75 

0.25 

0.75 

0.75 

4 

3 

2 

0.81179615 

43 

0.75 

0.75 

0.75 

0.75 

0.75 

0.75 

4 

3 

0.345 

0.61722312 

44 

0.25 

0.75 

0.25 

0.25 

0.75 

0.25 

1 

3 

0.345 

0.78397653 

45 

0.25 

0.75 

0.25 

0.75 

0.25 

0.75 

4 

3 

2 

0.8092077 

46 

0.75 

0.25 

0.25 

0.75 

0.25 

0.75 

1 

3 

2 

0.86006426 

47 

0.25 

0.75 

0.75 

0.25 

0.75 

0.25 

1 

6 

2 

0.57391456 

48 

0.25 

0.75 

0.25 

0.75 

0.75 

0.25 

4 

6 

2 

0.59651631 

49 

0.75 

0.75 

0.25 

0.5 

0.75 

0.25 

4 

6 

0.345 

0.5381402 

50 

0.75 

0.75 

0.25 

0.75 

0.25 

0.5 

1 

3 

0.345 

0.78542539 
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Run 

%  Man 

%  Man 

%  Man 

%  Man  ES 

%  Man 
Update 
Tracker 

%  Man 

#  Men 

#  Men 

Machine 

Resulting 

BBG 

NBG 

Visual 

TMA 

TMA 

Pool 

Effectiveness 

Score 

51 

0.25 

0.75 

0.5 

0.25 

0.25 

0.25 

4 

5 

2 

0.75746317 

52 

0.25 

0.25 

0.25 

0.5 

0.75 

0.75 

4 

5 

2 

0.62885725 

53 

0.75 

0.75 

0.25 

0.75 

0.5 

0.75 

3 

6 

2 

0.49936421 

54 

0.25 

0.75 

0.75 

0.75 

0.75 

0.25 

3 

6 

0.345 

0.53913317 

55 

0.25 

0.5 

0.25 

0.75 

0.25 

0.25 

1 

5 

1.1725 

0.76869368 

56 

0.25 

0.25 

0.25 

0.25 

0.75 

0.25 

4 

6 

1.1725 

0.57334446 

57 

0.25 

0.25 

0.25 

0.25 

0.5 

0.5 

1 

3 

2 

0.87816763 

58 

0.75 

0.25 

0.5 

0.25 

0.75 

0.25 

4 

5 

0.345 

0.63839886 

59 

0.75 

0.25 

0.25 

0.25 

0.75 

0.75 

1 

3 

0.345 

0.77131623 

60 

0.75 

0.75 

0.75 

0.75 

0.25 

0.5 

4 

5 

2 

0.69745856 

61 

0.5 

0.75 

0.25 

0.5 

0.25 

0.5 

1 

6 

2 

0.68621551 

62 

0.25 

0.75 

0.25 

0.25 

0.5 

0.75 

4 

3 

0.345 

0.69669303 

63 

0.25 

0.25 

0.75 

0.25 

0.75 

0.75 

1 

6 

0.345 

0.46765684 

64 

0.25 

0.25 

0.75 

0.5 

0.25 

0.25 

4 

6 

0.345 

0.69840393 

Table  26  Input  Factors  and  DOE  input  value  matrix 
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The  results  for  sensitivity  to  the  input  factors  are  shown  in  Table  26.  The  analysis 
was  performed  to  find  those  factors  and  cross-factors  that,  at  the  95  percent  confidence 
interval,  are  not  sensitive.  Table  27  shows  the  sensitivity  analysis  of  only  those  factors 
that  have  a  probability  of  less  than  0.05. 


Term 

Probability>|t| 

#Men  Pool(3,6) 

<.0001* 

%Man  Tracker(0.25,0.75) 

<.0001* 

%Man  TMA(0.25,0.75) 

<.0001* 

Machine  Effectiveness(0.345,2) 

<.0001* 

%Man  NB(0.25,0.75) 

<.0001* 

%Man  Tracker*#Men  Pool 

<.0001* 

%Man  Tracker*%Man  Tracker 

0.0076* 

%Man  TMA*#Men  TMA 

0.0163* 

%Man  NB*Machine  Effectiveness 

0.0230* 

%Man  BB*%Man  ES 

0.0265* 

%Man  NB*#Men  Pool 

0.0336* 

#Men  TMA(1,4) 

0.0422* 

%Man  Vis*%Man  Vis 

0.0455* 

Table  27  Input  Factors,  Second  Order  and  Two  Level  Cross  Terms  that  demonstrate  sensitivity 


The  DOE  analysis  was  re-run  only  for  those  factors  and  cross-factors  shown  in 
Table  27.  The  resulting  analysis  is  shown  in  Table  28  and  Table  29. 
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Sorted  Parameter  Estimates 


Std  Error  t  Ratio 


#Men  Pool(3,6) 

%Man  Tracker(0.25,0.75) 

%Man  TMA(0.25,0.75) 

Machine  Effectiveness^. 345, 2) 

%Man  NB(0.25,0.75) 

%Man  Tracker*#Men  Pool 
%Man  Tracker*%Man  Tracker 
%Man  TMA*#Men  TMA 
%Man  NB*Machine  Effectiveness 
%Man  BB*%Man  ES 
%Man  NB*#Men  Pool 
#Men  TMA(1 ,4) 

%Man  Vis*%Man  Vis 

%Man  BB*Machine  Effectiveness 

%Man  ES(0.25,0.75) 

%Man  Vis(0.25,0.75) 

%Man  BB(0.25,0.75) 

%Man  ES*#Men  Pool 

%Man  BB*%Man  NB 

%Man  BB*%Man  TMA 

%Man  ES*%Man  Tracker 

%Man  TMA*Machine  Effectiveness 

%Man  TMA*#Men  Pool 

%Man  NB*#Men  TMA 

%Man  NB*%Man  ES 

#Men  Pool*Machine  Effectiveness 

%Man  Tracker*#Men  TMA 

%Man  Tracker*Machine  Effectiveness 

%Man  Vis*#Men  Pool 

#Men  Pool*#Men  Pool 

%Man  Vis*#Men  TMA 

%Man  TMA*%Man  TMA 

#Men  TMA*#Men  Pool 

%Man  BB*%Man  Tracker 

%Man  BB*%Man  BB 

%Man  ES*#Men  TMA 

%Man  Tracker*%Man  TMA 

%Man  NB*%Man  TMA 

%Man  Vis*%Man  Tracker 

#Men  TMA*#Men  TMA 

#Men  TMA*Machine  Effectiveness 

%Man  BB*#Men  Pool 

%Man  ES*%Man  TMA 

%Man  BB*#Men  TMA 

%Man  Vis*%Man  TMA 

%Man  ES*Machine  Effectiveness 

%Man  NB*%Man  Vis 

Machine  Effectiveness*Machine  Effectiveness 

%Man  Vis*Machine  Effectiveness 

%Man  NB*%Man  Tracker 

%Man  ES*%Man  ES 

%Man  BB*%Man  Vis 

%Man  NB*%Man  NB 

%Man  Vis*%Man  ES 


-0.106413 

-0.05237 

-0.04175 

0.027805 

-0.023145 

-0.025457 

0.0359093 

-0.010023 

-0.009458 

-0.009318 

0.0087327 

-0.008187 

0.0228399 

-0.007459 

-0.006369 

-0.006412 

-0.006086 

0.0048712 

-0.003956 

-0.003786 

0.0035239 

0.0033782 

0.0034914 

-0.003361 

0.0032943 

-0.003052 

-0.003068 

-0.002582 

0.0026868 

0.0073168 

-0.00236 

0.0046954 

-0.001745 

-0.001734 

0.0053553 

-0.001647 

0.0015577 

-0.001378 

0.001142 

-0.003036 

0.0009794 

0.0009424 

0.0007135 

-0.000698 

-0.000641 

0.0004589 

-0.000459 

0.0014069 

0.0003819 

-0.000353 

-0.000862 

0.000265 

-0.000835 

0.0001938 


0.003458 

0.003302 

0.003348 

0.003181 

0.003126 

0.003474 

0.010485 

0.0034 

0.003459 

0.003516 

0.003487 

0.00346 

0.009848 

0.0035 

0.00309 

0.003428 

0.003273 

0.003533 

0.003657 

0.003554 

0.003424 

0.00333 

0.003603 

0.003559 

0.003686 

0.003463 

0.003622 

0.003431 

0.003791 

0.011073 

0.003648 

0.009613 

0.003588 

0.003571 

0.011383 

0.003747 

0.003599 

0.003594 

0.003487 

0.01122 

0.003724 

0.003617 

0.003383 

0.003556 

0.003643 

0.003333 

0.003462 

0.012424 

0.003412 

0.003421 

0.010995 

0.003549 

0.011599 

0.003491 


-30. 
-15.56 
-12.47 
8.74 
-7.4) 
-7.3  5 
3.42 
-2.9  5 
-2.7  5 
-2.6  5 
2.50 
-2.3' 
2.32 
-2.1  5 
-2.05 
-1.8' 
-1.8  5 
1.38 
-1.0  5 
-1.0' 
1.03 
1.01 
0.97 
-0.94 
0.89 
-0.8  5 
-0.8  5 
-0.7  5 
0.71 
0.66 
-0.6  5 
0.49 
-0.4  3 
-0.4  3 
0.47 
-0.44 
0.43 
-0.3  5 
0.33 
-0.2' 
0.26 
0.26 
0.21 
-0.2) 
-0.1  5 
0.14 
-0.1  5 
0.11 
0.11 
-0.1  ) 
-0.0  5 
0.07 
-0.0' 
0.06 


<00 

<00 

<00 

<00 

<00 

<00 

0.00 

0.01 

0.02 

0.02 

0.03 

0.04 

0.04 

0.06 

0.06 

0.09 

0.09 

0.20 

0.30 

0.31 

0.33 

0.33 

0.35 

0.36 

0.39 

0.40 

0.41 

0.47 

0.49 

0.52 

0.53 

0.63 

0.63 

0.63 

0.64 

0.67 

0.67 

0.71 

0.75 

0.79 

0.79 

0.80 

0.83 

0.84 

0.86 

0.89 

0.89 

0.91 

0.91 

0.92 

0.93 

0.94 

0.94 

0.95 


Table  28  Input  Factors,  Second  Order  and  Two  Level  Cross  Term  Sensitivity  Analysis 


117 


Sorted  Parameter  Estimates 


Term 

#Men  Pool(3,6) 

%Man  Tracker(0.25,0.75) 

%Man  TMA(0.25,0.75) 

Machine  Effectiveness^. 345, 2) 
%Man  Tracker*#Men  Pool 
%Man  NB(0.25,0.75) 

%Man  Tracker*%Man  Tracker 
%Man  BB*%Man  ES 
#Men  TMA(1 ,4) 

%Man  TMA*#Men  TMA 
%Man  Vis*%Man  Vis 
%Man  NB*#Men  Pool 
%Man  NB*Machine  Effectiveness 


Estimate 

Std  Error 

-0.106813 

0.002631 

-0.050579 

0.002588 

-0.040332 

0.002702 

0.0287709 

0.002575 

-0.025886 

0.00279 

-0.022305 

0.002583 

0.0381427 

0.00803 

-0.009898 

0.002774 

-0.009087 

0.00261 1 

-0.008941 

0.002873 

0.0202512 

0.0076 

0.0073079 

0.002762 

-0.006186 

0.002709 

t  Ratio 


Pr< 

*.0( 

*.0( 

*.0( 

*.0( 

*.0( 

*:.0( 

*:.0( 

o.oc 

o.oc 

o.oc 

0.0' 

0.0' 

o.o; 


Table  29  Reduced  Input  Factors,  Second  Order  and  Two  Level  Cross  Term  Sensitivity  Analysis 


Using  JMP9®’s  Desirability  Maximizer  function,  the  values  shown  in  Figure  29 
were  determined  as  sensitive  to  the  input  factors. 
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#Men  Pool 


%Man  0.25  Machine  0.25 

Tracker  %Man  TMA  Effectiveness  %Man  NB 


1 

#Men TMA 


0.25 

%Man  BB 


0.75 

%Man  ES 


0.25 

%Man  Vis 


Figure  29  JMP  Sensitivity  Analysis 


Input 

Factor 

%  Man 
BBG 

%  Man 
NBG 

%  Man 
Visual 

%  Man 
ES 

%  Man 
Update 
Tracker 

%  Man 
TMA 

# 

Men 

TMA 

# 

Men 

Pool 

Machine 

Effectiveness 

DOE 

results 

25 

25 

25 

75 

25 

25 

1 

3 

2 

Table  30  Input  Factors  from  the  DOE 


It  is  noted  that  percent  machine  is  equal  to  1  minus  percent  man.  Table  30  shows  the  data 
set  that  provides  the  optimum  results  of  the  model  that  produce  an  average  utilization  of 
CCS  operators  at  34.1%. 


118 


XVI  APPENDIX  D:  UNTL  TASK  BREAKDOWNS 


The  UNTL  was  consulted  as  verification  that  the  various  identified  Combat 
Control  tasks  were  incorporated  in  the  concepts  and  simulations.  For  the  three  scenarios, 
the  task  list  was  consulted,  and  provided  Table  31  and  Table  32.  The  LO  functions  were 
supported  by  the  UNTL  task 
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LI  Capabilities  — > 

Detect 

Identify 

Trk 

Decide 

Engage 

L0  Functions  — *• 

Implement  Directed  Search 

Process  Acoustic  Data 

Process  Image  Data 

Process  ES  Data 

Process  AIS 

Assess  Acoustic  Data 

Assess  Image  Data 

Assess  ES  Data 

Fuse  Data 

Perform  TMA 

eS 

a 

U 

3 

Develop  SA 

5 

< 

1 

■5 

< 

< 

8 

t 

| 

z 

Cue  Sensors  for  Directed  Search 

From  UNTL  list 

NTA  1  DEPLOY/CONDUCT 
MANEUVER 

NTA  1.1. 2.3.1  Sail  Ship  from  Port, 
Anchorage,  or  Moorage 

NTA  1. 1.2.3. 2  Return  Ship  to  Port, 
Anchorage,  or  Moorage 

NTA  1.1.2.3.4  Conduct  Convoy 
Operations 

NTA  1.1. 2. 3.8  Conduct  Submerged 
Operations 

NTA  1.2.11  Conduct  Navigation 

NTA  1.2.12  Maneuver  in  Formation 

NTA  1. 5.4.1. 1  Conduct  Screen 

, 

1 

1 

, 

, 

, 

, 

1 

, 

1 

, 

, 

, 

! 

1 

3 

3 

II 

H 

II 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

1,23 

1,23 

1,23 

133 

133 

1,2,3 

133 

133 

1,23 

1,23 

1,23 

S 

S 

S 

S 

s 

S 

S 

S 

2 

2 

2 

133 

1,23 

S 

S 

S 

S 

s 

S 

S 

s 

S 

S 

s 

3 

3 

3 

s 

s 

s 

s 

s 

s 

S 

s 

BUI 

MM 

1,23 

NTA  1. 5.4.1. 2  Conduct  Cover 

NTA  1.5.4. 1.3  Provide  Area  Security 
NTA  1.5.4.1.4  Secure  an  Area 

NTA  1.5.7  Conduct  Naval  Special 
Warfare 

NTA  2  DEVELOP 
INTELLIGENCE 

NTA  2.1.1  Determine  and  Prioritize 
Priority  Intelligence  Requirements 
(PIR) 

NTA  2.1.3  Conduct  Collection 
Planning  and  Directing 

NTA  2.3  Process  And  E3ploit 

Collected  Information  and 
Intelligence 

NTA  2.3.1  Conduct  Technical 
Processing  and  E3ploitation 

NTA  2.4.1  Evaluate  Information 

NTA  2.4.2  Integrate  Information 

NTA  2.4.3  Interpret  Information 

NTA  2.4.4.3  Evaluate  the  Battlespace 
Environment 

NTA  2.4.4.4  Evaluate  the  Threat 

NTA  2.4. 4. 5  Determine  Enemy 
Courses  of  Action 

NTA  2.4.5  Prepare  Intelligence 
Products 

NTA  2.4.5. 1  Provide  Support  to  the 
Commander's  Estimate 

NTA  2.4. 5.2  Provide  Intelligence  to 
Develop  the  Situation 

NTA  2.4. 5. 3  Provide  Indications  and 
Warning  (l&W)  of  Threat 

s 

s 

s 

s 

s 

s 

S 

s 

■a 

El 

23 

El 

23 

23 

MSM 

s 

s 

s 

s 

s 

s 

S 

s 

23 

23 

23 

23 

23 

2,3 
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s 

s 

s 

s 

s 

s 

S 

s 

3 

3 

3 

3 

3 

3 

3 

s 

s 

s 

s 

■ 

s 

S 

s 

2 

2 

2 

2 

2 

2 

■ 

s 

s 

s 

s 

s 

s 

S 

s 

133 

2 

133 

133 

s 

s 

s 

s 

s 

s 

S 

s 
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2,3 

23 

23 

23 

3 

s 

s 

s 

s 

s 

s 

s 

s 

13 

13 

13 

13 

13 

13 

s 

s 

s 

s 

s 

s 

s 

s 

13 

13 

13 

13 

13 

13 

s 

s 

s 

s 

s 

s 

s 

s 

133 
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s 

s 

s 

s 

s 

s 

s 

s 

133 

133 

133 

1,2,3 

133 

133 

s 

s 

s 

s 

s 

s 

s 

s 

3 

3 

3 

3 

3 

3 

s 

s 

s 

s 

s 

1 

1 

1 

S 

S 

S 

1 

s 

s 

s 

s 

s 

s 

s 

s 

s 

s 

3 

1,3 

1,3 

1,3 

3 

, 

1 

s 

s 

s 

s 

s 

s 

s 

s 

S 

S 

3 

3 

3 

3 

3 

s 

s 

s 

s 

s 

s 

s 

s 

S 

S 

3 

3 

3 

3 

3 

s 

s 

s 

s 

s 

s 

s 

s 

S 

s 

3 

3 

3 

3 

3 

NTA  2.4. 5.4  Provide  Intelligence 
Support  to  Force  Protection 

NTA  2.4. 5.5  Provide  Intelligence 
Support  to  Targeting 

NTA  2.5  Disseminate  and  Integrate 
Intelligence 

s 

s 

s 

s 

s 

s 

s 

s 

s 

s 

2 

2 

2 

2 

2 
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1 

1 

! 

1 

, 

1 

1 

, 

, 

13 

13 

3 
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3 

3 

3 

3 

Table  31  UNTL  Breakdown  (Part  1) 
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NTA  3  EMPLOY  FIREPOWER 

NTA  3.1.2  Select  Target  to  Attack 

LEJ 

S 

S 

S 

S 

s 

S 

s 

S 

S 

mm 

mm 

13 

13 

NTA  3.1.3  Select  Platform(s)  and 
System(s)  for  Attack 

s 

S 

S 

S 

s 

s 

s 

S 

1 

! 

i 

1 

1 

! 

NTA  3.1.4  Develop  Order  to  Fire 

S 

1 

mm 

mm 

13 

BIB 

NTA  3.1.5  Conduct  Tactical  Combat 

Assessment 

s 

S 

8 

$ 

s 

H 

H 

■ 

■ 

2 

B 

2 

■ 

NTA  3.2. 1.1  Attack  Surface  Targets 

1  S  1 

s 

S 

S 

S 

s 

m 

3 

3 

3 

3 

3 

3 

NTA  3.2. 1.2  Attack  Submerged 

Targets 

s 

S 

S 

S 

s 

3 

3 

3 

3 

3 

3 

NTA  3.2.5. 1  Conduct  Command  and 
Control  (C2)  Attack 

H 

s 

S 

S 

S 

s 

■ 

■ 

■ 

■ 

■ 

■ 

13 

13 

13 

NTA  3. 2. 8. 3  Engage  Targets 

1  s  1 

s 

S 

S 

S 

s 

m 

un 

1  S  1 

13 

13 

13 

3 

NTA  3. 2. 8.4  Adjust  Fires 

s 

S 

S 

H 

H 

3 

3 

3 

3 

NTA  3.2.9  Conduct  Nonlethal 
Engagement 

H 

s 

S 

S 

S 

s 

H 

■ 

■ 

m 

■ 

■ 

1 

1 

1 

NTA  4  PERFORM  LOGISTICS 

AND  COMBAT  SERVICE 
SUPPORT 

NTA  5  E3ERCISE  COMMAND 

AND  CONTROL 

NTA  5. 1.1.1  Transmit  and  Receive 

Information 

S 

s 

S 

S 

S 

s 

S 

s 

s 

s 

S 

1,2 

3 

3 

3 

NTA  5. 1.1. 1.1  Provide  Internal 

Communications 

s 

s 

S 

S 

s 

s 

s 

s 

s 

s 

S 

1,2 

3 

3 

3 

NTA  5.1. 1.1.2  Provide  E3ternal 

Communications 

s 

s 

s 

S 

s 

s 

s 

s 

s 

s 

s 

1 

3 

3 

3 

NTA  5. 1.1. 1.2.1  Receive  and 

Transmit  Force  Orders 

s 

S 

3 

3 

3 

NTA  5.1.2  Manage  Means  of 
Communicating  Information 

2 

2 

3 

3 

3 

NTA  5. 1.3.1  Maintain  and  Display 
Tactical  Picture 

2,3 

2,3 

2,3 

2,3 

2,3 

2,3 

23 

23 

1,2,3 

1,2,3 

1,2,3 

1,2,3 

1,2,3 

1,23 

1,23 

NTA  5. 1.3. 2  Maintain  and  Display 
Force  Command  and  Coordination 

Status 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

NTA  5.2  Analyze  and  Assess 

Situation 

S 

S 

S 

S 

S 

1,2,3 

1,2,3 

1,2,3 

1,2,3 

1,2,3 

1,2,3 

1,2,3 

1,23 

1,23 

NTA  5.2. 1.1  Review  and  Evaluate 

Situation 

S 

S 

S 

S 

S 

3 

3 

3 

3 

3 

13 

13 

1,2,3 

1,23 

1,2 

1,2 

NTA  5.2. 1.2  Review  and  Evaluate 

Mission  Guidance 

S 

S 

S 

S 

S 

1,23 

NTA  5.2. 1.3  Review  Rules  of 
Engagement  (ROE) 

s 

S 

s 

1 

S 

3 

NTA  5.2.2  Decide  on  Need  for  Action 
or  Change 

S 

S 

13 

13 

13 

13 

13 

NTA  5.3  Determine  and  Plan  Actions 
and  Operations 

S 

S 

S 

S 

S 

S 

S 

S 

S 

S 

S 

S 

1,2 

1,2 

2 

2 

NTA  5.3.2  Issue  Planning  Guidance 

S 

S 

3 

3 

3 

3 

3 

NTA  5.3.4  Analyze  and  Compare 
Course  of  Action 

S 

s 

S 

S 

1,2,3 

1,23 

NTA  5.3.5  Select  or  Modify  Course  of 
Action 

S 

s 

S 

s 

1,2,3 

1,23 

NTA  5.6  Conduct  Acoustic  Warfare 

1,2,3 

1,2,3 

1,2,3 

1,2,3 

1,2,3 

1,2,3 

1,23 

1,23 

1,23 

1,23 

Table  32  UNTL  Breakdown  (Part  2) 
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